All the unit tests in the world, the largest QA team still can’t stop bugs from slithering into production. With a distributed microservice architecture debugging becomes much harder. Especially across language & machine boundaries. APMs and logs provide the first steps. But in these crucial moments we need something more.
Production bugs are the WORST bugs. They got through unit tests, integration tests, QA and staging… They are the spores of software engineering. We cross our fingers and put on the Sherlock Holmes hat hoping that maybe that bug has somehow made it into the log… When it isn’t there our only remedy is guesswork of more logging. That in turn bogs performance for everyone, makes the logs damn near unreadable. Elastic helps, but there are limits on everything. We have no choice other than crossing our fingers and going through CI/CD again and again until we find it.
There are better ways. With continuous observability tools we can follow a specific process as it goes through several different microservices and “step into” as if we were using a local debugger without interrupting the server flow. In this session I will demonstrate such an approach and how everything integrates into elastic nicely.
