CLOUD APP DEVELOPMENT
Article | February 18, 2022
Our security research team prepared to explain a real attack scenario from the black box and white box perspective on how a vulnerable AWS Lambda function could be used by attackers as initial access into your cloud environment. Finally, we show the best practices to mitigate this vector of attack.
Serverless is becoming mainstream in business applications to achieve scalability, performance, and cost efficiency without managing the underlying infrastructure. These workloads are able to scale to thousands of concurrent requests per second. One of the most used Serverless functions in cloud environments is the AWS Lambda function.
One essential element of production raising an application is security. An error in code or a lack of user input validation may cause the function to be compromised and could lead the attackers to get access to your cloud account.
About AWS Lambda function
AWS Lambda is an event-driven, serverless compute service which permits the execution of code written in different programming languages and automates actions inside a cloud environment.
One of the main benefits of this approach is that Lambda runs our code in a highly available compute infrastructure directly managed by AWS. The cloud provider takes care of all the administrative activities related to the infrastructure underneath, including server and operating system maintenance, automatic scaling, patching, and logging.
The user can just use the service implementing their code and the function is ready to go.
Security, a shared pain
From a security perspective, due to its nature to be managed by the cloud provider but still configurable by the user, even the security concerns and risks are shared between the two actors.
Since the user doesn’t have control over the infrastructure behind a specific Lambda function, the security risks on the infrastructure underneath are managed directly by the cloud provider.
Using AWS IAM, it’s possible for the user to restrict the access and the permitted actions of the lambda function and its components. Misconfiguration on permission over IAM roles or objects used by the Lambda function might cause serious damage, leading attackers inside the cloud environment. Even more importantly, the code implemented into the Lambda function is under user control and, as we will see in the next sections, if there are security holes into the code, the function might be used to access the cloud account and move laterally.
We are going through two attack scenarios using two different testing approaches: black box and white box testing, which are two of the main testing approaches used in penetration testing to assess the security posture of a specific infrastructure, application, or function.
Looking at the Lambda function from a different perspective would help to create a better overall picture of the security posture of our function, and help us better understand the possible attacks and the related risks.
Black box vs white box
In Black box testing, whoever is attacking the environment doesn’t have any information about the environment itself and the internal workings of the software system. In this approach, the attacker needs to make assumptions about what might be behind the logic of a specific feature and keep testing those assumptions to find a way in. For our scenario, the attacker doesn't have any access to the cloud environment and doesn’t have any internal information about the cloud environment or the functions and roles available in the account.
In White box testing, the attacker already has internal information which can be used during the attack to achieve their goals. In this case, the attacker has all the information needed to find the possible vulnerabilities and security issues.
For this reason, white box testing is considered the most exhaustive way of testing. In our scenario, the attacker has read-only initial access in the cloud environment and this information can be used by the attacker to assess what is already deployed and better target the attack.
In this attack scenario the attacker found a misconfigured S3 bucket open to the public where there are different files owned by the company.
The attacker is able to upload files into the bucket and check the files configuration once uploaded. A Lambda function is being used to calculate the tag for each file uploaded, although the attacker doesn’t know anything about the code implemented in the lambda.
We can be pretty confident there is an AWS Lambda function behind those values. The function appears to be triggered when a new object is created into the bucket. The two tags, Path and Size, seem to be calculated dynamically for each file, perhaps executing OS commands to retrieve information.
We can assume the file name is used to look for the file inside the OS and also to calculate the file size. In other words, the file name might be a user input which is used in the OS command to retrieve the information to put in the tags. Missing a user input validation might lead an attacker to submit unwanted input or execute arbitrary commands into the machine.
In this case, we can try to inject other commands into the file name to achieve remote code execution. Concatenating commands, using a semicolon, is a common way to append arbitrary commands into the user input so that the code would execute them if the user input isn’t well sanitized.
We have seen the attack scenario from the black box and white box perspectives, but what can we do to mitigate this scenario? In the proposed scenario, we covered different AWS components, like S3 buckets and AWS lambda, in which some security aspects have been neglected.
In order to successfully mitigate this scenario, we can act on different levels and different features. In particular, we could:
Disable the public access for the S3 bucket, so that it will be accessible just from inside and to the users who are authenticated into the cloud account.
Check the code used inside the lambda function, to be sure there aren’t any security bugs inside it and all the user inputs are correctly sanitized following the security guidelines for writing code securely.
Apply the least privileges concept in all the AWS IAM Roles applied to cloud features to avoid unwanted actions or possible privilege escalation paths inside the account.
Let’s have a look at all the points mentioned above in detail on how we can enforce those mitigations.
Disable the public access for the S3 bucket
An S3 bucket is one of the key components in AWS used as storage. S3 buckets are often used by attackers who want to break into cloud accounts.
It’s critical to keep S3 buckets as secure as possible, applying all the security settings available and avoiding unwanted access to our data or files.
For this specific scenario, the bucket was publicly open and all the unauthorized users were able to read and write objects into the bucket. To avoid this behavior, we need to make sure that the bucket is available, privately applying the following security settings to restrict the access.
CLOUD APP DEVELOPMENT
Article | July 18, 2022
How Kubernetes and Linkerd became Lunar’s multi-cloud communication backbone
At Lunar, a Scandinavian online bank, we embraced cloud native tech early on. We’ve been running Kubernetes since 2017 and today have over 250 microservices distributed across three clouds.
This blog will explore how we set out to centralize all platform services. The gains were substantial — from being better prepared to absorb newly acquired companies to improved developer productivity.
Founded in 2015, Lunar set out to challenge the banking status quo by reinventing how people interact with their finances. Lunar is for those who want everything money-related in one place — 100% digital, right in their hands.
For us, that meant offering customers a smarter way to manage their money with more control, faster savings, easier investments, and no meaningless fees. That’s how we envision the future of banking.
In 2021, Lunar acquired Lendify, a Swedish lending company; and PayLike, a Danish fintech startup. This is all part of Lunar’s broader strategy to grow and scale. It also meant we had to integrate all these systems, so they work together smoothly.
Lunar’s commitment to cloud native principles
Lunar’s team of 150+ full-time engineers push about 40 releases to production on any given day. Out of these 150, ten are platform engineers, and that’s the team that I lead.
We operate nine Kubernetes clusters across three cloud providers (AWS, Microsoft Azure, and Google Cloud Platform) on multiple availability zones. We also run 250+ microservices plus a range of platform services that are part of our self-service developer platform. We want our teams — or Squads as we call them — to be autonomous and self-driven. To support this “shift left” mindset, a group of platform Squads builds abstractions and tooling to ensure developers can move their features fast, securely, compliant, and efficiently.
The Lendify acquisition means we now have an Azure-based platform we have to integrate and adapt, so it complies with the same cloud native principles Lunar is built on. We are currently working on seamlessly connecting our AWS and Azure environments.
There are multiple reasons why we chose the cloud native path. First, we needed a platform that allowed our teams to manage their services and be fully autonomous. Secondly, as a fintech company pioneering cloud-based banking, we had to provide a clear exit strategy for cloud providers — a regulatory requirement by the Danish FSA.
Kubernetes was a perfect fit. Functioning as an abstraction on top of a cloud provider, it helped us achieve both goals.
This autonomy allowed us to scale easily as most dependencies were removed. Squads are also supported by a mix of open source tooling, including Backstage, Prometheus, and Jaeger, and some custom-built solutions, which we have open-sourced, such as shuttle and release-manager.
This multi-cloud strategy and work style support the company’s goal of scaling, both in terms of the number of employees and mergers and acquisitions. It also allows us to stay technology agnostic and choose the technologies that best fit our needs.
Oh no, where are our production logs?
The idea of centralizing platform services started with our log management system Humio. At the time, we were developing failover processes for our production Kubernetes clusters. As it turned out, this led to missing logs in our log management system. That’s when we realized we had to remove the system from our production cluster and centralize it before performing any failover in production.
From logs to centralizing all platform services
After successfully centralizing our log management system, we decided to embark on a platform services centralization journey prior to any corporate acquisitions. While we had multiple environments, many of our platform services, such as our observability stack, were replicated in each environment. These services require a vast amount of resources and are fairly complex. Services such as Humio, Prometheus, and Jaeger (with Elasticsearch), are stateful services. Having stateful services in “workload” clusters makes failover and disaster recovery much harder. For this reason, we decided to minimize the number of stateful services in these environments. Additionally, running nine replicated setups, simply didn’t scale — we needed a centralized solution.
Moreover, having multiple endpoints for accessing things like Grafana, led to lots of duplication of users, dashboards, etc. This caused some confusion for our developers, changes had to be made in multiple places, leading to drift between environments, and other challenges. Managing users in one system was a lot more efficient than doing so in nine (or more).
That’s why we decided to create a centralized cluster owned by the platform team that would eventually run the entire observability stack, release management, developer tooling, and cluster-API.
Today, our log and release management runs as centralized services the platform team provides. Also, Backstage is provided out of the centralized environment along with a handful of other tools. Next in line is our monitoring setup, a mix of Buoyant Cloud and Prometheus/Grafana.
The quest to connect our clusters
Once we started centralizing platform services, we needed to connect our clusters. At the time, we were only running clusters in AWS and considered VPC peering across our accounts. Doing that was somewhat painful due to clashing CIDR ranges. We also evaluated VPNs but aren’t big fans of using technologies with two static boxes on each end. Besides, we wanted to move towards zero trust networking, following the principles of BeyondProd by Google.
Service meshes finally caught up with our needs!
We continuously evaluated service meshes during our 5+ years of running Kubernetes in production. In 2017, we had Linkerd running as a PoC but decided against it. It was still the JVM-based Linkerd 1 and quite complex. We kept following the development and evolution of service meshes and, when we saw the Linkerd 2.8 release and its multi-cluster capabilities, we realized it was time to give service meshes another shot.
Our decision was further reinforced by some problems we were experiencing with gRPC load balancing (which is not natively supported by Kubernetes) and the need to switch to mTLS for all internal communication. A service mesh made a lot more sense now.
While we evaluated both Linkerd and Istio, we have always been big fans of the approach Linkerd took: start with the basics and make that work well. We gave ourselves a week: two engineers; one playing with Istio and the other one with Linkerd.
We had the Linkerd multi-cluster up and running within an hour! After a few days of struggling with Istio, we gave up on it. Linkerd did the job fast and easily — the perfect mesh for us. It had all the features we needed at the time; was easy to operate, had a great community, and solid documentation.
Since going live, we also started using Buoyant Cloud for better visibility across all our environments.
Lunar is committed to the CNCF stack
At Lunar, we are big fans of CNCF projects and use many of them (in fact, I’m a CNCF Ambassador and love educating the community on these awesome projects!). Lunar is also a CNCF End User Member.
Our stack includes Kubernetes, Prometheus, cert-manager, Jaeger, Core DNS, Fluent-bit, Flux, Open Policy Agent, Backstage, gRPC, and Envoy among others. We’ve built an Envoy-based ingress/egress gateway in all clusters to provide a nice abstraction for developers to expose services in different clouds.
Prepared to scale our business and shake up the European banking market
From a technology perspective, we have now achieved a fairly simple way to provide and connect clusters across clouds. Kubernetes allows us to run anywhere, Linkerd enables us to seamlessly connect our clusters, and GitOps provides an audited way to manage our environments across multiple clouds with the same tooling and process. And from a developer perspective, whether you deploy on GCP or AWS, the process is identical.
Seamless integration with newly acquired startups
The business impact has been substantial. With our new multi-cloud communication backbone, we are better positioned to support upcoming mergers and acquisitions — a key part of our business strategy. Having a cloud agnostic way to extend the Lunar platform regardless of where they run, is incredibly powerful. It also allows us to select the provider that best fits our needs for each use case.
Fully prepared for DR while compliant with government regulations
The fact that we are no longer losing logs during failover is huge. We’ll soon implement quarterly failovers for our production clusters. We need to ensure we know exactly how our system behaves in case of a failure and how to bring it back up. It’s important both from a regulatory perspective and a business perspective. If our customers were to lose access to their account information, it would have disastrous consequences for our business. That’s why we proactively train for the worst-case scenario. If something were to happen, we would know exactly what to do and how to avert an issue.
We are big believers in the pets vs. cattle idea but go a step further. We don’t want to have pet servers or pet clusters either. Imagine losing logs each time we perform a failover. Without audit logs, we’d fall out of regulatory compliance right there and then.
Centralized services and streamlined processes increased developer productivity
Centralizing most of our platform services has already streamlined many processes and improved developer productivity. We ensure that all releases, metrics, logs, traces, etc., are properly tagged with fields such Squad names, environments, and so on, making it easy for developers to find what they are looking for. It also ensures clear ownership of that particular piece.
Managing the team is also a lot simpler. For me, that means I don’t have to set up dashboards, help search through logs, etc. — our Squads are truly independent. Because our platform is based on self-service, it is decoupled from the organization allowing our team to focus on implementing the next thing that will help our developers move faster, be more secure, or ensure better quality.
Easy audits and peace of mind for management
Then there are the easy audits. Since everything is centralized, we can run audit reports for all clouds and services across groups and environments. That is good for us and provides peace of mind in the highly-regulated financial services industry.
While we aren’t there yet, we expect to save significant time in engineering resources by not having to operate and maintain nine versions of the soon-to-be fully centralized stack.
Well-positioned to scale fast and smoothly
Overall we feel well-positioned for upcoming acquisitions and organic growth. With a platform able to extend anywhere, we’ve become a truly elastic organization.
CLOUD APP DEVELOPMENT
Article | May 20, 2022
Elastic announced the launch of Elastic Security for Cloud, extending the capabilities of the existing Elastic Security offering, which included SIEM and endpoint protection, to incorporate cloud risk and posture management and cloud workload protection.
The new solution allows enterprises to manage their security posture for cloud-native and hybrid environments with infrastructure detection and response (IDR) and a machine learning offering that can detect known and unknown threats in cloud environments.
This approach means that enterprises can detect and respond to malicious activity in the cloud as soon as possible to minimize the risk and damage caused by intrusions.
Addressing cloud complexity
The release comes as modern enterprises are struggling to secure cloud environments.
Research shows that not only are 76% of organizations using two or more cloud providers, but also that 35% of organizations have more than 50% of their workloads in the cloud. This means for many organizations, protecting the cloud attack surface is now vital to maintaining operational stability.
In other words, the complexity of cloud deployments has created a need for solutions with cloud detection and response capabilities, so that enterprises can respond to threat actors who are targeting this new attack surface.
“The world has rapidly transitioned to the cloud and chosen operations over security. Cloud infrastructure is stood up and torn down at a blazing rate and many different teams are deploying these cloud instances. Just answering the question, ‘Am I good?’ is an increasingly complex question to answer for CIO/CISOs,”
Mike Nichols, vice president of product for Elastic Security
Elastic’s answer to monitoring these environments, is a cloud posture management and IDR solution that can improve security teams visibility over external threats.
The cloud security market
Elastic Security is competing in the global cloud security market, which researchers valued at $40.8 billion in 2021 and anticipate will grow to $77.5 billion by 2026 as enterprises attempt to keep up with the increasing sophistication of cyberattacks, and rise in bring your own device (BYOD) and (choose your own device) policies.
In the realm of cloud security, one of Elastic’s main competitors is Splunk, a cloud and observability monitoring platform that can monitor public clouds, apps, services, on-premise data centers, and edge services. Last year, Splunk announced it had received a $1 billion investment from Silver Lake.
Another competitor in the market is Wazuh, an XDR and SIEM tool designed for protecting endpoints and cloud workloads with vulnerability detection, log data analysis, workload protection, and container security.
Wazhu currently has more than 10 million downloads per year and is used by a range of companies including Verifone, Walgreens, Rappi, Grubhub, Intuit, and more.
Although, one of the key differentiators between Elastic Security is that the solution is built on the Elastic Search Platform, with analytics, SIEM, endpoint protection, XDR and cloud security all offered as part of one single offering.
CLOUD APP DEVELOPMENT
Article | May 20, 2022
Database Management in the Cloud Computing Era
A cloud computing database is ideally a service that is built, deployed, and delivered via a cloud platform. A cloud platform as a service (PaaS) delivery model allows organizations, end-users and applications to store, manage and retrieve data using the cloud. When seen from a structural and design perspective, a cloud database is not very different from one that operates its business using its own on-premise servers. However, ever since big data has entered space, database management has become a little more complex. In addition to all the traditional and structured data, we also have semi-structured and unstructured data coming in from almost all directions.
In recent times, there has been significant adoption of cloud platforms, infrastructures and services. The idea of the blend of cloud technology with database services has garnered more demand with respect to management.
Cloud databases, which arealso widely termed Database-as-a-Service (DaaS), offer various added options for organizations to choose. With the current rate of adoption, experts have a strong feeling that DaaS will, just like any other "as-a-service", will become the standard solution for all highly sensitive and mission-critical data.
How Can Cloud Database Management Help Your Business?
Every organization has the constant need to manage its data in the most efficient way possible. A cloud database effectively fulfills all the needs of any organization with respect to data, ranging from keeping the information secure, accurate and consistent for resource utilization and optimal performance. Cloud data management is constantly changing the way organizations think about data. The cloud helps bring in the required versatility, security, and professional data management assistance that is required. For any business to survive and succeed, it should ensure that the data is healthy so that everyone in the organization has access to the data they need, when they need it.
"Line-of-business leaders everywhere are bypassing IT departments to get applications from the cloud (also known as software as a service, or SaaS) and pay for them similar to a magazine subscription. And when the service is no longer required, they can cancel that subscription with no equipment left unused in the corner." - Daryl Plummer, Gartner analyst
Effective Strategies forDatabase Management in Cloud Computing
Using a cloud-based database ensures that it is easy for your database to grow along with your needs and requirements, in addition to scaling up or down on-demand to accommodate specific peak-workload phases. Ideally, before procuring a cloud data management system, it is essential to have a solid strategy that would suit best with your organization's ecosystem and, at the same time, would also help you getthe most out of the system you select.
Exploring the Best Practices for DBM in Cloud Computing
With the idea ofeffectively developing, monitoring and managing database infrastructure, there are various methods that organizations can use. These days, organizations also have the option for a ready-made database management system or picking a tailored solution as per their requirements. Keeping all these aspects in mind, it is also essential that these organizations enroll themselves in the best practices followed to ensure optimum results are achieved.
Some of the best practices for database management in cloud computing are mentioned below:
Before moving to the cloud, build a robust data management architecture.
Give cloud data integration requirements the first priority.
Regardless of the data's platform or location, govern it comprehensively.
Use encryption and VPN to protect data in transit.
Automate database management tasks to keep track of them.
Why Is Cloud DBMS the C-Suite’s Pick?
While it is more than evident that in today's age and time, data management is one of the most crucial components of any organization, it is also right up the area of value addition for all the top executives in an organization. Every industry relies vastly on data and its management, with the significant shift towards cloud servicestaking place constantly several organizations and the c-suite are now evaluating what cloud services bring to their table, especially on the data front. Once the core pain points are assessed and the c-suite gets the picture of how cloud database management systems would help them become more efficient, there wouldn't be any other obvious choice.
Microsoft Consulting Services, along with Tata Consultancy Services (TCS), partnered with Walgreens to create the Information, Data, and Insights (IDI) platform powered by Azure.When patients or medical providers initiate the prescription fulfilment process, Azure Databricks and Azure Synapse Analytics move hundreds of related data points through the IDI. Pharmacists can access information through a centralized dashboard interface and request data visualizations.Pharmacists gain real-time insights, and the system can scale as needed to meet current needs. During peak times, the platform can handle more than 40,000 transactions per second. Compared to Walgreens’ previous solution, the private cloud-based architecture, saves an enormous amount of time on every transaction — prescription data that once took about 48 hours to reach its data warehouse can be handled by Walgreens within minutes.
With the next big things being data and the cloud, how can an amalgamation of both be wrong? Having said this, everyone willhave their own share of concerns and questions. But with what is being built and functions being worked upon, this space is bound to bring in numerous opportunities. So when the world is slowly makingthis shift, it's time you reconsidered, if you haven't already, before you get overwhelmed at the helm of data!
What Are the Best Features Available in Cloud Database Management?
While there are numerous features that one can leverage using Cloud Database Management, the top 5 of its features are listed below:
Easy data reduction
Which Is the Most Popular Cloud Database?
One of the most popular and widely used cloud databases is the Microsoft Azure cloud database. It provides services in computing, networking, databases, analytics, artificial intelligence, and the Internet of Things.
In Cloud Computing, What Is Database Management?
Cloud data management is a technique for managing data across cloud platforms, either in combination with or instead of on-premises storage. Disaster recovery, backup, and long-term archiving can benefit from using the cloud as a data storage tier. In addition, resources can be acquired as needed using cloud data management.