terraform security testing

One more, a rather peculiar thing is that terrascan rule descriptions for GCP are pretty much identical to the ones used by Checkov. Very promising feature is running as an API server. They should answer the question, Does my configuration or plan contain the correct metadata? Traditionally, unit tests should run independently, without external resources or API calls. Run terraform plan again to create a new execution plan. Synchronize HCP Consul services as backend address pools for Microsoft Azure Application Gateway using Consul-Terraform-Sync (CTS). In this post, Ill discuss some testing strategies for HashiCorp Terraform configuration and modules so that you can terraform apply with greater confidence. of having fully-fledged functional tests that are mostly used for Before we set out to choose the Terraform security scanning tool that will best suit our needs, we checked other posts concerning themselves with this topic. If you are maintaining DevSecOps microservice pipeline, then it is the right tool for you. We make the cloud work for you. The tests cannot accurately represent the end state of brownfield (existing) resources because they do not test updates. Module testing applies mostly to immutable resources because of its create and delete sequence. Unit tests for the plan representation require Terraform to authenticate to your infrastructure provider and make comparisons. optional faster_parsing pip install flag in the Installation Guide. Fast-track to success with flexible on-demand cloud services. The module uses a dedicated Terraform Cloud workspace with an attached Sentinel policy of its unit tests. Negative testing is the process of ensuring that a system can gracefully handle unexpected input or unwanted behavior. You can use them to check for correct identifier formats, naming standards, attribute types, and value constraints such as character limits or password requirements. It misses impact severity and security background as in other tools in our selection. Checkov uses Bridgecrew's API to enrich the results with links to remediation guides. Frameworks like Terratest and kitchen-terraform can also be used for end-to-end tests. Monitor feature of the Snyk agent is good for SCA but does not work on Terraform code well. similar functionality only for terraform while it is free-to-use and it is Open Source. What is needed here is oversight to ensure conformity to stated rules. I explained the different types of tests and how you can apply them to catching errors in Terraform configuration before production, and how to incorporate them into pipelines. Configure Terraform: If you haven't already done so, configure Terraform using one of the following options: Terraform-compliance tool: Install the Terraform compliance tool by running the following command: pip install terraform-compliance. Using HTML report generator for snyk in code directory example: The harder part of this task was to fairly compare the selected tools based on the quality of the security findings. They create a load balancer, listener rule, and target group to verify that the module configures additional listener rules and target groups. Huge comparison of cloud providers for the gaming industry. For example, use a custom validation rule to ensure that an AWS load balancers listener rule receives a valid integer range for its priority. Upon merging the changes, my GitHub Actions workflow runs integration tests written in Terratest. Youll learn how infrastructure tests fit into your organizations development practices, the differences in testing modules versus configuration, and approaches to manage the cost of testing. When you test Terraform modules, you want enough verification to ensure a new, stable release of the module for use across your organization. Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support. Once deployed, youll be able to explore several violations of infrastructure security best practices such as: We welcome teams to fork the TerraGoat repo and practice troubleshooting and debugging bad infrastructure. If you are running dockerized pipeline you can run tfsec in docker, and not only tfsec. For larger, more complex configurations with many resources, you should not manually check individual parameters. This directory is referred to as the example directory. If you do not use HashiCorp Sentinel, you can use your programming language or configuration testing tool of choice to parse the plan representation in JSON and verify your Terraform logic. In addition to custom validation rules, you can use Terraforms rich language syntax to validate variables with an object structure and check that the module receives the expected input. Terrascan clones your repository code into the container and scans it. You could, but it may not be the best use of your time and effort! Since they run independently of infrastructure resources, unit tests have a virtually zero cost to run frequently. Unit and contract tests may require extra time and effort to build, but they allow you to catch configuration errors before running terraform apply. At the next level from the bottom of the pyramid, contract tests check that a configuration using a Terraform module passes properly formatted inputs. Another person then applies a different version of the code that provisions different version of that VM. Operations team members should still be able to examine system metrics in their monitoring tools. End-to-end tests can verify that changes did not break expected functionality. Doing compliance checks is straight forward. The top half of the pyramid includes tests that require active infrastructure resources to run properly. Terraform enables the definition, preview, and deployment of cloud infrastructure. terraform also provides the This also fixes the pro, Simplify support for running module directly (. Terragoat is a vulnerable-by-design Terraform project designed to give DevOps engineers a place to learn how to identifying misconfigured infrastructure as code modules and test infrastructure code without polluting their own professional AWS account. Secure example feature is a very good approach for showing secure code. Tfsec is having quite good remediation and recommendation details with links to AWS, GCP or Terraform documentation. More about this topic, maybe, in one of our next posts. Since you have to set up and tear down the resources, you will find that integration tests can take 15 minutes or more to complete depending on the resource! In the AWS load balancer case, add a map representing service objects and their expected attributes and type. The first policy could be written as a BDD feature scenario as follows: The following code shows a test for a specific tag: In this section, you download and test the example. Interesting feature is error history which can show you when the engineering errors were introduced during the resource lifetime. Run terraform show to convert the execution plan to JSON for the compliance step. After the integration tests pass, I can tag and release a new version of the module. Check out the prototype. A module delivery pipeline starts with a terraform plan and then runs unit tests (and if applicable, contract tests) to verify the expected Terraform resources and configurations. Ability to be run by engineers on demand. The configuration is now in compliance with the policy. You can use a known host file to define the connectivity to GitLab or GitHub via ssh. To start with the easier part we decided to gather the features of the compared tools based on our requirements and also add other ones that we thought could be interesting or useful. Compliance testing is done through the inspection process and the outcome of the review process should be documented. Depending on your integration testing framework, you may need to write scripts or automation to terraform apply for test resources, run the tests, and terraform destroy the resources. Finding is referenced via a range of lines where you need to look for specific attributes in contrast with snyk. These checks are defined as simple JSON or YAML so you do not need to write additional GO code. The pyramid offers a common language to describe what area a test can cover to verify configuration and infrastructure resources. Since Terraform is mostly used against Cloud APIs, we still need a way to ensure the code deployed against the infrastructure follows specific policies. You can terraform validate and terraform plan to check your configuration, but will that be enough? Ensure all data stored in the RDS is not publicly accessible, Ensure no hardcoded AWS access key and secret key exists in EC2 user data, Ensure no security groups allow ingress from 0.0.0.0:0 to port 22, Ensure no hardcoded AWS access key and secret key exists in lambda environment, S3 Bucket has an ACL defined, which allows public READ access. Depending on your systems cost and complexity, you can apply a variety of testing strategies to Terraform modules and configuration. Everybody can find benefits in different tools based on specific needs and mainly integrations. Are you puzzled from the wide range of choices of static analysis tools for Terraform? Run terraform validate to validate the syntax of the configuration files. Sign up for our monthly newsletter to stay up to date with all things Bridgecrew. If possible, avoid creating entire systems. To manage the cost, run integration tests after merging feature branches and select the minimum number of resources you need to test the module. For example, you might define geopolitical naming conventions for your Azure resources. Coincidence? At Bridgecrew, were firm believers that cloud security needs to happen as early as possible-ideally when infrastructure is planned and applied using code. We also believe that DevOps, security, and engineering teams strive to do this. Position of the finding (defined by filename; resource; code line or line range in which the issue was found). They answer the question, Does this module or configuration create the resources successfully? A terraform apply offers limited integration testing because it creates and configures resources while managing dependencies. This section shows how to translate that example into a BDD Feature and Scenario. For example, your Terraform configuration parses the IP address from an AWS instances DNS name and passes it to a target group for a load balancer. End-to-end tests usually depend on an entire system, including networks, compute clusters, load balancers, and more. Learn HashiCorp tools with self-guided tutorials, videos, and hands-on labs. If you are not skilled enough you can use policy builder via UI, which is very intuitive and offers connection to several benchmarks and standards like HIPAA, CIS, NIST. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Another reason we built TerraGoat is to support the development of and the community around Checkov. Terraform security is becoming an increasingly important area for DevOps engineers to learn and implement. You can use terraform fmt -check and terraform validate as rudimentary unit tests. The import of the repository is very easy and you just need to define your GitLab API key scopes to give read only access to the selected repository. Conduct module tests in a different project or account so that you can independently track the cost of your module testing and ensure module resources do not overwrite environments. In an ideal way, this Scenario (among with all other Scenarios) will run on a CI/CD pipeline that will ensure that nothing is deployed by violating your policies. At the bottom of the pyramid, unit tests verify individual resources and configurations for expected values. Were excited to release TerraGoat , a training project for developers to learn about how to identify and avoid infrastructure as code misconfigurations and security risks. Compliance testing is a nonfunctional testing technique to determine if a system meets prescribed standards. The unit tests do not need to cover the configuration in modules. Infrastructure as code (sometimes referred to as infrastructure code or abbreviated as IaC) is used, As companies look inward to discover where they can shore up cloud security shortcomings, the, Today we are introducing Checkov, a new open-source project for cloud infrastructure security. Marko Fbry, Cloud ArchitectMarek ottl, Cloud Security Engineer. Check out TerraGoat on Github to get started. Learn testing strategies for HashiCorp Terraform modules and configuration, and learn how to run tests against infrastructure. You can use the same testing framework as your unit tests to check that a Terraform configuration passes the right inputs to a module. Checkov offers automated remediation or manual fixes. To check that youve upgraded Nomad properly, you can deploy a sample application, test the endpoint, and delete it from the cluster. When choosing a framework, consider the existing integrations and languages within your organization. Another interesting thing to notice, is that all four tools were performing relatively similarly on the AWS files, and there were lots of issues that were found by three (7) or even all four tools (6), while in the GCP files there were only 4 issues on which three tools agreed and there was no issue which would be discover by all four tools. The module contains contract tests in the form of variable validation, which will verify valid inputs for any configurations that depend on the module. Checkov supports running only / skipping specific checks: Checkov is also offering a paid version. Ability to filter specific rules / ignore specific findings (mitigating false positives or accepting risk), 7. Worth mentioning is PR commenter which adds a comment to any area of the code which fails the tfsec scan. If youre not familiar, Checkov is the open-source static analysis tool for infrastructure as code that we released earlier this year. Using the VM example, a compliance policy might be as follows: "If you're creating an Azure resource, it must contain a tag". At the very least, automate some tests to reduce the time you need to manually verify changes and check for errors before they reach production. These types of tests overlap with security testing done as part of policy as code because you check attributes in Terraform configuration for the correct values. capability Terrascan has Notifier providing webhooks for the results. Some companies can benefit from paid versions due integrations and reporting options. The HCL syntax allows you to specify the cloud provider - such as Azure - and the elements that make up your cloud infrastructure. The main motivation for this task was to find the best tool to include in our infrastructure pipeline that will help us identify security issues in Terraform code defining AWS and GCP infrastructure. They also communicate the expected values of configuration across your team and organization. The test will fail because - while the first rule requiring existence of tags succeeds - the second rule fails in that the. This was done by installing the tools, experimenting with them, checking their websites, code repositories, documentation, command line help, outputs, etc. Compliance testing would be used to enforce rules in these and many other scenarios. If you need Software composition analysis, Kubernetes configuration scans etc., then it is a very good choice. To skip this API call you can use the flag. Once you define the policies, a tool like Terraform-compliance is used to ensure the policies are followed. Please do not create an GitHub issue for security concerns. terraform-compliance is providing a To solve this problem, write two unit tests with HashiCorp Sentinel to check parameters in the configurations plan and automate the IP address verification. When testing modules, consider the cost and test coverage of module tests. However, none of these tests verify correct variable interpolation, list iteration, or other configuration logic. without having any encryption. The difference between development and production will affect the outcome of your tests, so be aware of which tests may be more important to flagging errors or disruptive to run. for your case. Should you verify every parameter that Terraform configures on a resource? Of course, this is just an example which may or not be applicable For additional test coverage, you can use any programming language or testing tool to parse the Terraform configuration in HCL or JSON and check for statically defined parameters, such as provider attributes with defaults or hard-coded values. To check that the monitoring tool has system metrics, you can check if it contains data from your system in the last five minutes. Compliance testing is also known as conformance testing. As a result, you usually run a scaled-down version of production to save cost. Checkov provides very easy to run scanning over repo directory with possibility of your own checks. You see the dashboard with errors by policy or failures by benchmark. Manual testing involves spot-checking infrastructure for functionality and has a high cost in time and effort. To ensure sufficient test coverage, write unit, contract, and integration tests for modules. The Terraform-compliance tool provides a test framework where you create policies like the example. Coordinating infrastructure as code configuration best-practices across multiple engineering teams is difficult, not to mention that not every development squad has the right DevOps security skills. for service, service_data in var.services : aws_lb_target_group_attachment_does_not_use_node_address, aws_lb_target_group_attachment_has_ip_address, "^\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}$", target_group_attachment.values.target_id not in, consul_terraform_sync_service_node_addresses, target_group_attachment.values.target_id matches, "Priority of listener rule between 1 to 50000", "The priority of listener_rule must be between 1 to 50000. Then we changed the shape of the findings usingjqso that every issue finding contained: After that we concatenated all the results, sorted them by filename, resource, tool and issue finding ID. Because. It does not provide links to advanced help or examples of good coding practice. If you need to report a non-security related bug, please open an issue on the Terraform GitHub repository. proving a component of code is performing properly. Among the issues that Snyk found and checkov did not, were things like fully open egress or missing description of AWS security groups or load balancer facing the internet. We also looked at the remediated findings in the final report. We deeply appreciate any effort to disclose vulnerabilities responsibly. The goal of BDD is to encourage teams to build concrete examples that express a common understanding of how the system should behave. These stakeholders generally include the developers, testers, and anyone with a vested interest in - or who will be impacted by - the system being developed. that creates/updates/destroys entities. Complete guide for picking the right tool for Terraform Security Code Analysis. After running integration tests, destroy the resources and release a new module version. You should write additional tests to check for configuration parameters on the active resource. What is your choice? With Lambda AWS container image support it starts to be very interesting for native DevSecOps serverless scanning. Contract tests answer the question, Does the expected input to the module match what I think I should pass to it? Contract tests ensure that the contract between a Terraform configurations expected inputs to a module and the modules actual inputs has not been broken. Whether youve updated some HashiCorp Terraform configuration or a new version of a module, you want to catch errors quickly before you apply any changes to production infrastructure. Terraform-compliance - a free and open-source tool - provides this functionality for Terraform configurations. There are very few readily available examples of what bad infrastructure code looks like, and there arent any resources out there to help train and test your infrastructure as code skills. a lightweight, security focused, BDD test framework against terraform. The checks are written in python so some coding skills are needed in comparison with tfsec. You ran terraform plan to manually check the IP address and continue to add more configuration to the module over time. The higher up the pyramid you go, the fewer tests you should have for that level of the pyramid. Integration tests check that a configuration using a Terraform module passes properly formatted inputs. PART 2: AWS, AZURE or GCP? Unlocking the Cloud Operating Model: Thrive in an era of multi-cloud architecture. We bring you cloud technologies adapted to your needs, with rapid time-to-value and innovative solutions. As a result, it takes time to scroll through the planned changes to check your IP address. For more information about using Terraform in Azure, see the Azure Terraform developer center. Terrascan performs very poorly in definitions of the tasks and remediation description. ", "Consul services monitored by Consul-Terraform-Sync", this repository with a list of infrastructure testing tools, Test-Driven Development for Infrastructure, Moving Security and Sanity Left by Testing Terraform with InSpec, Terraform Code Reviews: Supercharged with Conftest, Testing Infrastructure as Code on Localhost, Number of resources or attributes generated by, Values associated with interpolated values, Expected variables or outputs marked as sensitive. In its ideal form, a long-running development environment that accurately mimics production can help you catch potential problems. 4. Failed changes to active environments will affect critical business systems. Ability to add and develop your own security rules, 8. You can write end-to-end tests in any programming language or framework. How do you know if you can run terraform apply to your infrastructure without affecting critical business applications? We testedcheckov,snyk,terrascanandtfsec. The report references the exact line, resource and attribute in Terraform, which we consider as perfect for speeding up the code review. We apply best practices and focus on security to make sure we can maintain the trust of the community. There is also an option of using Snyk broker in case you run a private repository. Auto-fix is part of the paid version. With compliance testing, there are two important concepts to consider: compliance testing and compliance checks. Instead, focus on unit testing any configuration not associated with modules. Terrascan also underperforms on GCP terraform code. Previously in this article, you read about a compliance-testing example of creating a VM for a test environment. Run terraform apply to apply the execution plan. Idea of API server and container can be merged in usage of AWS EKS (alternatively GKE) or ECS. Expecting all cloud developers to know all those best practices is a little unrealistic. If you do not want to use public reporting and maximise the output enrichment, you can choose to utilise snyk-to-html. to ensure everything is up-to-date between the local configuration and the remote API(s) responses. The workflow runs unit tests written in Sentinel against a Terraform Cloud workspace. Important additional feature is the ability to create custom checks. Working in the gaming industry as a game developer, CTO, CEO, game analyst or any other role is Keep up to date with the latest news and updates! Instead, unit and contract tests quickly automate the verification of important configurations and set a foundation for collaboration across teams and organizations. The output of each phase is compared against the documented requirements. Fuzzing is an example of negative testing. With Checkov you can scan the growing list of vulnerable resources that Terraform provisions into an AWS account. In reality, your tests may not perfectly align with the pyramid shape. They answer the question, Can someone use the infrastructure system successfully? For example, application developers should still be able to deploy to HashiCorp Nomad after you upgrade the version. In the same vein as RhinoLabs fantastic penetration testing training tool, CloudGoat, TerraGoat is a complementing resource for Terraform, the popular infrastructure-as-code framework from HashiCorp. Ill start at the bottom of the pyramid with unit tests and work my way up the pyramid to end-to-end tests. The motivation behind this was to unify different preferences of engineers at Revolgy in order to provide improved and more consistent secure services to our customers. For a full example of testing a module in Terraform Cloud, refer to a module built for Consul Terraform Sync. The new HashiCorp Terraform Azure Stack provider 1.0 includes several provider behavior improvements, 19 new resources, and supports a new service for Key Vault. From a practical standpoint, you may not always have a development environment that fully replicates a production environment because of cost concerns and the difficulty of replicating user traffic. When you focus purely on Terraform files, the performance is very good but the description of the issues is very vague and missing links to documentation of AWS/ GCP/ Terraform. Unfortunately, theres a major knowledge, skills, and access gap between those functions, making it difficult to achieve. Fortunately, Terraform is an abstraction layer for any API that creates, updates, or destroys cloud-infrastructure entities. Within the example directory, navigate to the src directory. Development Principles. The rule is first expressed using Cucumber, which is a tool used to support BDD. Be sure to this project on GitHub, and keep an eye out for whats coming next, CfnGoat, a vulnerable-by-design training project for CloudFormation. Therefore we didnt include formatting and linting tools liketflintin this comparison.We also avoided testing frameworks such asconftest,kitchen-terraform,terrafirma,terraform-complianceorterratest. Also when you will look for your SAST tool for IaC, ask yourself if you also want to test kubernetes configs, open source libraries and docker images. Terrascan is also available as GitHub action. A resource group is created with a name following the pattern. But what we want to raise is known_hosts file for Terrascan in Docker container. Run docker run again to test the configuration. Spinning up databases and clusters can take half an hour or more. You see a message indicating that the Terraform configuration is valid. The results of the test are any "gaps" in not conforming to the predetermined standards. 2022 Bridgecrew inc. All Rights Reserved. In production, run end-to-end tests against the system to confirm system availability. As a result, implement as much unit and contract testing as possible to fail quickly on wrong configurations instead of waiting for resources to create and delete. Additionally, unit tests can validate: Overall, unit tests run very quickly and provide rapid feedback. Broker proxy is critical if you need to use public API and private code management. At a glance, you dont know if it correctly replaces the hyphens and retrieves the IP address information. Write unit, integration, and end-to-end tests for configuration of active environments. Integration tests can check that changes successfully run in a long-lived development environment, and end-to-end tests verify the environments initial functionality. One way to address this issue would be to define a policy of tagging the resources - such as with role and creator tags. Revamped whole Security Groups functionality. Use Checkov pre-commit hooks or linters to identify errors before getting pushed into run-time, identify common missing arguments, and explore security layering concepts like security groups and IAM hardening. You signed in with another tab or window. Terrascan supports around 500 policies similarly as other tested tools. When youre constantly pushing changes, you might even create multiple test instances! A set of standards and procedures is developed and documented for each phase of the development lifecycle. After you apply your Terraform changes to production, you need to know whether or not youve affected end-user functionality. Configuration parsing, terraform fmt -check, and terraform validate do not require active infrastructure resources or authentication to an infrastructure provider.