To preface, I think it would help to provide context on where I work and a bit about the team. I am the Director of Engineering at Brandfolder, where we enable the world’s largest brands to centralize, protect, and distribute their brand and brand assets on our digital asset management platform. Our team is highly engaged and aggressively building what we believe to be the future of DAM. To do this we lean heavily on enabling technology and product advancements, but try to maintain a level of consistency that provides us the maintainability needed for all of our products.
Our architecture can be defined as a Monolithic Core with supporting Microservices. We embrace the development velocity and centralization of context we get from our monolith, but recognize that functional services or 3rd party vendors can compartmentalize commodity-like services which we can plug into the existing architecture. The challenge here is how do we empower new technological choices that can move the needle forward for us without compromising our ability to maintain the systems we’ve built?
Before we answer that question, it’s important to consider our goals and values are as an engineering team: Agility - shipping features, product feedback loops, development cycle Visibility - bug feedback loops, change management, product usage/data analytics Collaboration - ability to pick up a new repo, context sharing, shared visibility and ownership Security - credentials, exposure, compliance
The current iteration of our answer to this challenge is one that fits these goals while providing immense freedom for our team to explore and leverage the latest technology to solve problems and ship product. We have a consistent development process across all of our repos through considerations like the 12Factor App, feature branch with pull requests workflow, and repo-level test coverage. We also structure our github repo configurations identically where-by we enable github vulnerability checks, protected master branches, squash merge strategies, and required status checks. All of these are verified by repo crawling from the github API. Our next component of the software lifecycle stipulates that all code is deployed via our Continuous Integration system, of which we have tremendous love for CircleCI and the flexibility it provides us.
Not all repos look the same, nor do all apps execute the same, but as a developer you can expect all of our repos to deploy the same: via the continuous integration system. Create a Pull Request, pass the required status checks, and merge to master to trigger a deploy :rocket:! Last, but not least, we also place a large importance on application monitoring and visibility. These requirements are satisfied via a few traditional methods: log aggregation into a single application log sink, error reporting into the appropriate error monitoring system, and application metric capture.
Circling back to the “chaos”, the importance of these standards increases as we diversify the deployed environments for our code. A quick rundown of some places our code is “deployed” to: Flynn PaaS Kubernetes Serverless (AWS Lambda, GCP Cloud Functions) Single Page Applications (blob storage with CDN proxy) 3rd party execution environments (Scraping hub, etc.) Software Library Packages GCP Managed Services (Composer, DataProc, etc.) Software Plugins: (Chrome, GSuite, etc.) As you can see, our applications are running in all sorts of places. Each of these environments was chosen because it met the requirements for scalability, execution, or flexibility. We have found great success leveraging these various tools to complete targeted tasks and functions throughout our application.
To further emphasize the need for such a method for software and requirements to accompany, let’s see a few quick stats on our code footprint: - 42 deployed applications - Over 1M Lines of Code - 9 Languages - Avg >10 Deployments/day
Amidst the constant change and various repos we use, we’ve found a relative consistency in our Software Development LifeCycle. I hope for any other teams reading this you can find inspiration in some simple patterns that make code easy to approach and repositories easy to contribute to without adding process for process sake. We believe in empowering our team to make the largest impact we can have on our clients and their users’ experiences. For each situation we evaluate what type of solution makes the most sense and fit the result into our system complicit with our SDLC.