November 6, 2017

Understanding Query Plans

I was recently trying a product feed for all products on the Spring website for a 3rd party vendor. The implementation plan was as follows: Create a cron job that runs once a day to create a product feed. The product feed is a CSV file in S3 which contains information such as product name, description, brand name, photo urls, taxonomy, etc. Only active products should be included in the feed. Read more

November 6, 2017

Things to Account for in a Project Estimation

Experience with shipping backend projects Deployments What could go wrong? Bug Bash Phase Usually at least a week Requirements you don’t consider Setting Up Logs & Monitors Datadog Logging metrics to Kibana Database Migration Issues Database timeout issues Database can’t handle the load from the new project Setting Up New Services They will take longer than expected

November 6, 2017

Building a Product Catalog with Elasticsearch

What is Elasticsearch? Search engine based on Lucene NoSQL database Communicates over HTTP Terminology Index: (equivalent to a SQL database) Type: (equivalent to a SQL table) Mapping: Specifies what fields we expect each document to have Document: Installing Elasticsearch brew install elasticsearch && brew info elasticsearch Defining the Index and Mapping (Technically an Optional Section: Defining the mapping is actually optional because Elasticsearch is schema free by default, but it is better to be more explicit about what is being stored. Read more