Imported from a Blogger post I wrote in 2008. Code shared on GitHub

For some reason I seem to find time to write code in R when I’m on an airplane. The last two trips I made resulted in significant enhancements and debugging of the code for my headroom plot. It started off simple but it now has a lot of bells and whistles, including color coding.

Main changes: the quantile used to remove outliers now only removes outliers that exceed the 95th percentile response time by default. It keeps all the throughput values unless you use qx=True.

In each…

I’ve recently become involved with the new Linux Foundation Open Source Climate Finance organization (OS-Climate). I’m helping manage AWS contributions to the project, as we build an open source data lake and analysis service that can be used to model climate related asset risks for investors. This includes physical asset risk from things like sea level rise and extreme weather events, as well as market changes from customers deciding to buy more sustainable products, and economic risk as the world economy is impacted by climate change. …

One of the impacts of COVID-19 has been a large rapid unplanned change in business activity for most of the global economy. There are several outcomes, some markets like work from home tools and services, healthcare and grocery retail are extra busy, social media applications are extra busy but their revenue from advertising may be impacted, and many businesses in manufacturing, travel and entertainment are have largely been shut down because their workforce and customers are in lockdown. While datacenter based IT costs are largely fixed, in a three year depreciation schedule, cloud costs are inherently variable, and can be…

When I joined Netflix in 2007 I was managing a team that built the personalized home page for the DVD shipping web site. The first thing I found was that every user visible change we made went through an A/B test. This was institutionalized across Netflix from the beginning, and it’s part of the formation story told in Marc Randolph’s book That Will Never Work. As an experienced product manager and the founding CEO, he wanted to be able to measure what was really working. An oft-quoted study by Microsoft Bing found that of the changes they test, one-third prove…

March 17, 2020

A resilient system continues to operate successfully in the presence of failures. There are many possible failure modes, and each exercises a different aspect of resilience. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks. In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. Failing silently represents a much bigger risk than when the same failure is clearly and…

We’re excited to let you know that we have an Open Source track at re:Invent this year! Learn about what’s in it: https://aws.amazon.com/blogs/opensource/open-source-at-aws-reinvent-2019/ Some selected sessions: OPN201 Change is coming to robotics development: The shift to ROS 2 🤖 Alejandra Olvera-Novack — AWS Developer Relations on how the shift from Robot Operating System (ROS) 1 to ROS 2 will change the landscape for all robot lovers. OPN220 Build robotic cloud simulations with ROS and AWS RoboMaker Join Camilo Buscaron, AWS Principal Open Source Technologist, and Katherine Scott, Developer Advocate, Open Robotics in this workshop to use Gazebo, a 3D simulator…

A resilient system continues to operate successfully in the presence of failures. There are many possible failure modes, and each exercises a different aspect of resilience. The system needs to maintain a safety margin that is capable of absorbing failure via defense in depth, and failure modes need to be prioritized to take care of the most likely and highest impact risks.

In addition to the common financial calculation of risk as the product of probability and severity, engineering risk includes detectability. Failing silently represents a much bigger risk than the same failure that is clearly and promptly reported as…

I recently read Annaka Harris’ new book Conscious about the mysteries and nature of consciousness and had an epiphany. What we do to make our computer systems observable is actually closely related to consciousness. The essential quality of conciseness is a model of the system. That model is fed with information and interrogated about the overall health and behavior of the system. If the system is ourselves then we can ask “how do you feel?”, “do you need anything?”, “what did you learn today?”, “what do you think is going to happen next?”, “What is the solution to this problem?”…