Google Warsaw Cloud Meetup

This evening belonged to Google Cloud's amazing Technical Support team, and they talked about solving complex and not so obvious problems our customers need help with.

Here's the link to the recording.

Providing highly-available and reliable locking service for a loosely-coupled distributed system is not a trivial problem to solve and it gets even more challenging at Google scale. Emilia talked about our experience with working on such a lock service (Chubby), shared how Paxos implementation with lease timers fits into this picture and ensures liveness. Moreover she discussed mechanisms for scaling that are applicable for our purpose such as caching and proxies.

Here's link to the recording.

Additional reading materials:

1. The Chubby lock service for loosely-coupled distributed systems - https://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf
2. Paxos Made Live - An Engineering Perspective - https://research.google/pubs/pub33002/

In this talk, Jan took a deeper look at the infrastructure behind Google's home-grown planet-scale replication service, discussed the challenges, and the approach taken to ensure that your data is geographically distributed for durability, availability, and latency purposes.

Here's the link to the recording.

Quite a lot of high profile mobile apps nowadays employ the use of various server-driven UI approaches to update not just content but layout & features without updating the app itself. This greatly empowers teams, as more engineers can directly contribute to the mobile apps but it adds an extra burden as development typically requires setting up both worlds on the engineer’s machine (mobile and backend infrastructure) or going through minutes-long build-deploy to pre-prod environments wait times to test their work.

We aim to show how one can solve this problem by shrinking the long build-deploy cycles to seconds and have almost instant feedback of the local code changes directly on one’s mobile phone.

Here's the link to the recording.

Google Cloud Data Analytics products store large amounts of structured and unstructured data organized in datasets, tables, files etc. Finding relevant resources in data lakes is a tough problem that Dataplex Search is addressing. In this presentation Witold explained why metadata search in GCP is different from Google Search. He also discussed how to efficiently index and search technical and business metadata and what challenges our product is facing in terms of scalability, security and freshness of the data.

Here's the link to the recording.

Writing postmortems after incidents and outages is an essential part of Google's SRE culture. They are blameless, widely shared internally, and allow us as an organization to maximize the insights from failures. We touch on how postmortems are written and used at Google, as well as how they can help in making decisions and driving improved reliability. We also show how you can get started with your own lightweight postmortem process.

Here's the link to the recording.

Alla, Nafisa and Santosh have walked us through a brief history of how design became a strategic partner at Google and changed over time to the way we know it today. They also looked deeper into the team processes and structures and how different types of designers play an essential role in product development in the real example.

Here's the link to the recording.

Diagnosing networks in native cloud environments is not trivial. This is mainly due to the scale and global nature of the network. In Google Cloud Platform a set of tools called Network Intelligence Center come to the rescue.

Nailia and Przemek presented the VPC architecture built on the Software Defined Networking (SDN) technology developed by Google. They also exploreed Network Topology - a tool that their teams created to better understand network traffic flows. They also touched on the architecture and engineering challenges they faced while building this tool.  

Here's link to the recording.

Service Mesh is becoming a key component in the Cloud Native world. It allows Dev and Ops teams to connect, secure, and observe applications without mixing business logic with infrastructure concerns. This way teams can focus on delivering value, letting the mesh do all the complex non-functional work like service discovery, load balancing, encryption, authentication, authorization, support for the circuit breaker pattern, and other capabilities. Istio is one of the major Open Source Service Mesh options available today.

In their talk, Paweł and Abdelfettah explained what Istio is, what problems it solves and how it plays nicely with GKE.  

Here's the link to the recording.

Eventarc is a product that helps to build event-driven architecture without having to implement, customize or maintain the underlying infrastructure.

Maciej and Sasha gave a quick overview of the monolithic vs event-driven architectures, explained how Eventarc can help with the latter, showed a short demo and walked everyone through the development process and explained how the Warsaw engineering team builds UIs like this, going from an idea to the launch.

Here's the link to the recording.

Wojciech explained how we scaled GKE and Kubernetes to enable the largest Internet Superstars to run their applications in multi-thousand node clusters. He also showed how we made that possible and what were the biggest challenges we had to overcome in both open source and internally.

Here's the link to the recording.

Cloud applications bring with it the opportunity to build regionally or even globally distributed systems. Such distributed systems come with technical challenges, such as ensuring that the systems are available even in the face of hardware or software failure. But they also have significant advantages, such as scaling on demand to a large number of customers. In this talk, we explore some fundamental architectures for Cloud applications.

We also take a step back and talk about why Google Cloud engineers work every day to make it easier for customers to build highly scalable and reliable systems, as more and more companies are moving to Cloud.

Here's the link to the recording.

In his presentation, Karol talked about the topic that interested him greatly, even before he joined Google: how does Google remain productive at such an unparallel engineering scale?

Google's engineering team is gigantic. Thousands of teams, engineers, and projects - a lot with their custom tech stack, needs, and preferences. What are the core building blocks that allow us to build new products in a complex and diverse environment? The answer lies within a few simple technologies that we're building on to not only achieve it but to do so productively. 

Here's the link to the recording.

Marcin explained what kind of unique challenges  need to be solved at Google's scale when we are  running many data centers, and in each of them many clusters of many thousands of servers each.

He talked how the scale at which Google’s workloads are running presents its specific challenges, and why addressing them in a cost-effective way is hard because at the other end of the system there are service owners who run their systems on Borg, and running even a simple service at that scale also turns out to be a tricky job.

Here's the link to the recording.

Aniket talked about Anthos, Google's brand-new Hybrid and Multicloud offering and why it is the UI that is at the center of this paradigm shift. He also explained the challenges we are facing in the UI team regarding scale, performance, security and authentication.

Here's the link to the recording.

Alex presented some of the unique challenges at Google, that are not often found in other organizations. This includes the way we work with our codebase, how we test, review and integrate changes towards the mainline. In this talk, you'll find out more about the key takeaways that led us to how Google engineers worldwide operate on a daily basis today - and how this can be useful for you. 

Here's link to the recording.

Filip presented challenges of running a reliable, scalable and easy to manage service in cloud and how GCP's autoscaling and workload management products can help. You also had a chance to peek under the hood and discuss interesting problems Google engineers are solving in order to build GCP products like these.

Here's link to the recording.

Our first event focused on sharing the work engineers in Warsaw do. Most of our engineers are working on Google Cloud Platform (or GCP as we call it - we love acronyms!), throughout the complex stack, on projects ranging from Google’s internal infrastructure (that the vast majority of Google products are dependent on) to designing and developing user interfaces for the majority of Google Cloud products. A lot of what we do is around scaling, and in this event we’ve shared an overview of the technical challenges engineers in Warsaw are facing. In other words - it is a scaling game, and in our upcoming events we’ll deep dive into specifics. The goal of these meetups is for everyone attending to learn at least one new thing about cloud engineering - hope you already have!  

Here's link to the recording.