Nicholas Comuni

Recommender System Architecture. Designed, built and maintain a recommender system serving millions of users, made up of several interoperating sub-models tied together by orchestration logic, business rules, feature engineering and data-transformation pipelines. Recently led the redesign that split the monolithic scoring pipeline into independent scoring and delivery stages, simplifying deployment and reducing the blast radius of failures.

Recommender System Data Model. Designed and maintained the full data model on Databricks Unity Catalog, including the feature store architecture and dozens of tables spanning training, inference, monitoring and orchestration metadata. Databricks Unity Catalog · Delta Lake · SQL · PySpark · LucidChart

Model Development & Release. Deployed several models to production across teams and stakeholders. Built a methodology for developing, testing, evaluating and releasing ML models on a champion/challenger architecture and MLOps best practices, automating releases and documentation. MLflow · Confluence · LucidChart

A/B Testing Models in Production. Designed and orchestrated the simultaneous deployment of multiple models to production, measuring real-world impact with business metrics to quantify lift/gain during online evaluation.

PySpark Pipeline & Table Optimization. Redesigned the feature store architecture for performance and cost, cutting compute cost/runtime by 90–95% (thousands of USD/month) across most workloads by analyzing Spark execution plans, eliminating costly operations, and applying Liquid Clustering, caching and partitioning strategy. Delta Lake · Databricks Unity Catalog · Liquid Clustering · Spark · Databricks Feature Store

Serverless Migration. Migrated all recommender system workloads to Databricks Serverless compute — resolving dependency conflicts and required code changes — reducing infrastructure cost and eliminating cluster-configuration overhead.

Retrieval Recommender Model. Designed and developed a two-tower neural network retrieval model based on user/product interactions. TensorFlow Recommenders · PySpark · Databricks · MLflow · Python

Feature Engineering & Feature Store. Built and maintain 30+ feature engineering pipelines and architected the shared feature store end-to-end: consistent partitioning/primary-key conventions, multi-database routing for environment isolation, incremental (vs. full-refresh) scoring, and reusable base classes shared across every recommendation engine.

Hive Metastore → Unity Catalog Migration. Led the migration of tables, models, files, code, metadata and experiments from Hive Metastore to Unity Catalog as part of a cross-team initiative — a sensitive, high-risk migration requiring extensive planning, cross-team coordination and precise scripting to validate parity before cutover.

Zero-Party-Data Real-Time Model. Built a real-time deterministic model using zero-party data: explicit user preference signals (e.g. preferred brand, category) processed to deliver real-time personalized recommendations reflecting that stated feedback. Spark Structured Streaming · Databricks Real-Time Pipelines · DynamoDB

Ranking Recommender Model. Built the ranking system at the core of the recommendation engine using user, product and user-product interaction features from web and mobile (via Adobe Analytics). Shipped several model families, including ALS (Collaborative Filtering), XGBoost and LightGBM. ALS · XGBoost · LightGBM

User Clustering Model (Cold Start). Developed an end-to-end unsupervised K-Means clustering model to recommend to users with no interaction history, ranking offers by cluster-level behavior rather than individual signal. Databricks Pipelines · PySpark · PySpark ML

Model Testing. Implemented integration testing for the recommender system using pre-computed dataframes and mock datasets to validate multiple pipeline stages end-to-end, via a dedicated 'test' feature tag that runs pipelines in test mode.

Evaluation Framework. Designed an end-to-end evaluation framework for recommender systems following OOP best practices, with data/parameter validation via Pydantic. Covers Precision@K, Recall@K, Average Precision, NDCG, Reciprocal Rank, Intra-List Diversity, Coverage, Novelty, Uniqueness, Serendipity and Average Percentile Rank. Pydantic · Python

Full Repository Refactoring. Redesigned entire repositories to MLOps best practices: modularized codebases, standardized project structure across repos, extracted a shared internal utilities library used by every recommendation engine, moved pipelines to config-driven (YAML) execution, and introduced consistent testing/CI.

Global Control Group. Designed and implemented a global control-group framework so every recommender pipeline can distinguish control-group users from the personalized-offers population, enabling accurate measurement of real incremental lift — requiring coordinated changes across every engine's scoring and delivery logic.

Automated Compliance Audit Pipeline. Built a scheduled pipeline that programmatically audits production jobs against best-practice standards (naming, tagging, compute policy, IAM usage, permissions, leftover dev artifacts) and publishes findings directly to Confluence. Parameterized so other teams can plug in their own rule sets. Built end-to-end with Claude Code. Databricks SDK · Confluence REST API · Python

Data Monitoring & Drift Detection. Designed a pipeline to monitor data drift using statistical tests (Kolmogorov–Smirnov, Chi-Square, Cohen's D, correlation) alongside visual plots and reports, keeping a human in the loop to decide on retraining or flag data inconsistencies. Databricks · Databricks Pipelines · PySpark · Seaborn · Matplotlib · SciPy

ML Model CI/CD. Designed and implemented a CI/CD pipeline for ML models on the Databricks ecosystem using Databricks Asset Bundles and Jenkins. Databricks Asset Bundles · Jenkins · GitHub Actions · Databricks Jobs API

Knowledge RAG Agentic System. Built a RAG system ingesting GitHub repos, Jira tickets and Confluence docs into Databricks Vector Search, consumed by a LangGraph/LangChain agent deployed via Databricks Apps. MLflow AgentServer handles monitoring and inference; responses pass through guardrails and are scored with MLflow GenAI metrics (Completeness, Groundedness, Fluency). Databricks Apps · Databricks Vector Search · Databricks Secrets · LangGraph · LangChain · MLflow

Databricks Development Agent. Built an agent that streamlines development on Databricks, integrating MCPs, codebases, custom skills/tools and a knowledge graph/context system. Uses the Databricks CLI/API to fetch data, interact with resources, coordinate with other agents (e.g. Databricks Genie), and take real actions within clearly defined scope and guardrails. Anthropic · Databricks CLI · Databricks Secrets · Databricks Jobs/Volumes API · databricks-sql-connector · MCPs · Skills

String Matching Algorithm. Matched product data extracted from receipts via OCR against data scraped from vendor websites. The data was highly noisy, requiring extensive cleaning and normalization; a combination of deterministic rules, string matching algorithms and regex-based fuzzy matching produced a clean, unified final table. PySpark · Regex · RapidFuzz

Fortuna. Designed and built Fortuna from the ground up — a personal SaaS for financial planning. A full-stack agentic app where users log income, expenses, investments and debts to generate dashboards, reports and scenario analysis. An LLM-powered agent interprets financial entries, projects future scenarios and performs dashboard operations from natural-language instructions. Supports sharing plans, multiple plans, and comparing scenarios. React · JavaScript · HTML · CSS · Python · Flask · LangGraph · OpenAI API · PostgreSQL · GitHub Actions · Terraform · AWS App Runner · AWS Secrets Manager · AWS VPC

Instagram Automation. Built a marketing automation tool for Instagram using automated bots to collect data via web scraping and GraphQL requests, deployed on Google Cloud Compute Engine with Cloud Storage. Python · Selenium · GraphQL · Google Cloud Compute Engine · Google Cloud Storage · Linux Debian

Programador Raiz — YouTube Channel. Run a YouTube channel teaching Python, Data Science and Data Engineering through practical, hands-on tutorials. youtube.com/@programador_raiz

Data Engineering Articles — Medium. Write data engineering articles on Medium, explaining Spark concepts clearly and didactically.

Spark: A Brief Introduction Spark: Parallelism & Partitioning Spark: Performance Optimization Spark: Memory Management

Kaggle Competitions. Competed in Kaggle challenges, from computer vision to recommender systems and regression.

Sorghum — 100 Cultivar Identification (FGVC 9) H&M Personalized Fashion Recommendations New York Taxi Trip Duration

Landing Pages and Other Applications. Developed a number of landing pages and websites over the years, including e-commerce sites and this exact portfolio page you're browsing, built from scratch as a static, single-page living CV with a light/dark theme. HTML · CSS · JavaScript starktechnologies.com.br nicholascomuni.com

Video Platform Subtitle Translation. Developed and automated a Python subtitle processing and translation workflow for thousands of educational videos on a video platform, using the Google Translate API to translate content at scale and significantly cut the manual effort required. Python · Google Translate API

Irrigation System Budget Automation. Designed a desktop irrigation-budgeting tool with custom Python libraries for hydraulic and irrigation calculations, integrated with the ERP's MySQL database for live pricing and stock. Generates a full bill of materials and ready-to-send PDF budget proposals with engineering data and costs. Python · Pandas · MySQL · Tkinter

Python Scripts & Data Analysis. Built scripts to automate data processing and cleaning, plus financial analytics reports using data from MySQL and ERP systems. Python · Pandas · Matplotlib · Seaborn · MySQL

Data Warehouse. Designed and implemented a dimensional data warehouse on SQL Server, integrating multiple sources with fact/dimension tables and a star schema. Built SQL ETL pipelines orchestrated via Azure Data Factory and stored procedures, feeding Power BI dashboards. MySQL · SQL Server · Azure Data Factory · Power BI

Power BI Dashboards. Designed financial reports and dashboards, helping stakeholders define relevant KPIs and reviewing metric calculations for accurate, meaningful reporting. Power BI · SQL

Project Management Application. Designed, built and maintained a Django project-management app used by the sales team to submit projects and budgets to the engineering team. Kept continuous stakeholder communication to gather requirements and drive improvements. Python · Django · Jinja · HTML · CSS · Heroku · SQLite

Hydroponic Systems Budget Automation. Built a budgeting automation tool for hydroponic systems, cutting average delivery time from 6 hours to 30 minutes (−92%) while keeping a human in the loop. Given project parameters, it generates a full bill of materials, calculates engineering variables (pressure, flow rate, irrigation sectorization), estimates labor and produces an estimated project cost. Python · Pandas · NumPy

Experiment Management Application. Built a Django app to manage ongoing experiments at Ballagro's experimental site (biotech treatments on plants, biological pest control). Deployed to Heroku with S3 for storage, Celery/Redis for background tasks, and Plotly for data visualization. Python · Django · Jinja · HTML · CSS · Heroku · SQLite · Amazon S3 · Plotly · Redis · Celery

Biotechnological Experiment Reports. Organized, monitored and supported the field application of biotech agricultural products (e.g. Beauveria bassiana, Trichoderma). Ran statistical analyses (ANOVA, Tukey's test) and produced reports on results and their significance. Python · Pandas · SciPy · Microsoft Excel

Soil Fertility Analysis Tool. Built a tool to process soil-fertility CSV reports at scale, comparing nutrient levels against crop requirements (soybeans, corn) and generating fertilizer/macronutrient recommendations, with analytical reports and visualizations. Python · Pandas · Matplotlib · Seaborn · Jupyter Notebook