Search & recommendation systems infrastructure: all-in-kubernetes, Scylla one love, Opensearch, Clickhouse (10+ TB), Postrsql (~1TB), Kafka (500k msg per sec), Redis (never again). Monitoring & alerting, nice dashboards. Business-critical web services. Here i did regular boring CI/CD with Giltab, Argocd during 1st month, and it's been growing very well since then. I helped bootstrapping ML-platform from scratch with the team: Dagster for data pipelines (Spark & GPU incl), ClearML for notebooks and experiments, s3 as cold storage, PostgreSQL & Scylla as online store Cool part is we did manage to move from YandexCloud's DataSphere & Dataproc managed services to no-vendor-lock home-grown ML platform with Spark & gpu-operator on k8s ready for heterogeneous infrastructure. Here i also picked up data-engineering stream: wrote data-pipelines for candidate models with PostgreSQL as main online store. Tuned queries & developers to keep it under around 1 ms in 95pp. I designed & implemented few real-time counters in feature-store, that allowed low-latency heavy parallel queries + back-in-time data layer for model training. Wrote few fastapi & management services for model serving etc. Another win was adding Scylla with Rust API on top for feature-store I also pushed the product and added a couple of must-have features myself using dad-style python programming techniques

Deploy & on-going support duties for custom-made Ceph-like storage system. Specifically, s3-subsystem in YandexCloud regions: ~50k rps, ~1k hw servers, heavy Nginx\Lua configs, many special ci/cd tools, 200 TB logs per day + coredumps, bug hunting. Left for startup

Full-stack devops responsibilities for multi-million tvl decentralized crypto-exchange. Also I led resource optimization & transition from public cloud provider to leased hardware. As a result, reduced infra costs 10 times. ~10k rps, low-latency java-based applications, many TB PostgreSQL clusters, Kafka, 100+ micro-services, blockchain nodes. All in self-hosted k8s clusters. Gitlab, Argocd

So internet shop decided to build own cloud like Amazon did AWS. Our cloud started as "Kubernetes but with Libvirt" + UI, little billing, authorization and all. Here I was responsible for CI/CD, development environments and minor components of the cloud like DNS, monitoring, authorization, toy drivers for Hashicorp tools Day-to-day duty was to make sure developers can comfortable spin the whole system on their laptops, and every commit returns end-to-end tests results in ~15 min in Gitlab Also managed production environments and kept an eye that nothing breaks too bad

Build and Release team. Here I was packaging various software, dealt with binary compatibility for C++ apps & a-thousand-modules Python profiles, rolled trains, patched ELF. Proposed and implemented simple system to continuously install ~100 GB of stuff to ~10k hw boxes. Embraced clouds, utilized Nginx, wrote shell scripts and some Python. Also I did a project to deliver big software on small machines. Fuse + redis + s3. Managed micro services in Kubernetes with Argocd and kept cloud costs minimal while handling few thousand rps on ingress. Did seamless failover and migrations for state-full apps (Kafka, Postgresql, Redis: advanced Ansible + deep planning)

Here I worked with Zing JVM build system and dev flow, and smooth components integration (LLVM). Moved the product from Perforce to Git mono-repo and all the good things. Built some automation on Jenkins, Buildbot. Powered the flow with AWS, in-house OpenStack. Pushed QA cycle to become isolated and be able to scale.

Helping with Hyperledger Fabric blockchain framework and all the stuff around it. Git, Docker, Vagrant, Jenkins. Java, Golang

Here I integrated Venafi Trust Protection Platform with Kubernetes' CertManager (https://github.com/Venafi/cert-manager), and wrote cli tool (https://github.com/Venafi/vcert), and did setup Jenkins to build & test all that. Technology stack: Golang, x509, Kubernetes, Amazon Web Services (AWS), Vagrant, Jenkins, Cucumber

Technology consultant for Margin and Risk system for EMEA, APAC and USA locations. Here I do barely legal stuff to make things running and keep users happy - help resolving ad-hoc production incidents - bugs hunting & problem solving - providing code analysis for analytics and business users - interacting with end-users, developers, DBA & Unix support teams - managing Dev and UAT environments - tools with Java, Scala, Bash, etc. Stack: GNU Linux & Sun Solaris, Sybase, Docker, Java, Flex, WebLogic, JSP, Shell, Perl, etc.

- Provide Operating systems environment support for Application/DB teams - Incident, problem and change management within SLA, team coordination - Automate routine tasks - machines should work, people should think - Out-of-office hours on-call engineer support - Provide training/knowledge transfer and expert level support for colleagues from different teams - IT infrastructure audits and proposals writing - Packaged applications and automated OS installation/setup - Completed design and implementation Puppet Environment over 100+ Linux servers; wrote work instructions and trained services support specialists Worked with ~1000 servers of Red Hat Linux / IBM AIX / HP-UX running Oracle on HA clusters with concurrent filesystems and so, ~100 nodes of PostgreSQL including big ones. Digged into networking

- Mail/DNS/Videoconferencing services installation & ongoing support - Infrastructure monitoring - Backup/restore setup - Various web-services development and support - Work-flow optimization systems deployment Here I did just everything

- Telecommunications billing deployment and data migration - Workflow systems support and development (CRM and billing systems integration) - Developed web-based customers' self-service office - Developed online video streaming service - Provided full-stack support of web-based services and tools - Provided Mail/DNS/Backup&Restore/Monitoring services support - Provided Linux-based systems administration - Provided 70 TB storage system support and hardware maintenance