A core part of our Security Engineering team's work is code security. We regularly conduct code reviews covering new microservices, features, and the core libraries used throughout our projects. During these code reviews, we rely on a combination of unit test mutations, custom tools, and open-source tools to identify potential issues.
One of our favorite tools is Semgrep. It allows us to write custom static analysis rules tailored to our codebase. Using our application security posture management (ASPM) technology, Arnica, we have seamless integration of Semgrep rulesets throughout all of our code repositories.
Earlier this year, we shared our experience using Semgrep and contributing to public Semgrep rule repositories. Today, we’re excited to share some of the rules we’ve developed for public use through our own repository. These rules are a subset of the custom rules we use internally to secure our own applications.
Rule Development
Over time, we’ve crafted numerous Semgrep rules tailored to our codebase, which have become an integral part of our static analysis program. For example, some rules ensure that safe, internally developed wrappers for file-handling functions are used instead of the standard Go versions of the same type. Other rules are on the lookout for bugs that might seem innocent, hiding quietly in the code and pretending to be harmless, but that could ultimately lead to significant vulnerabilities if left unchecked. For public release, we’ve adapted many of these rules to generalize their applicability beyond our own code.
We’ve grouped the published rules into several categories, each addressing a specific type of security concern.
Secure File Creation
The insecure-dir-creation rule identifies instances where files or directories are created with restrictive permissions but without first verifying whether they already exist. In Go, if a file or directory already exists, creating it again replaces the existing file while retaining its original permissions. This behavior can lead to security issues if the permissions of the existing file are less restrictive than intended, such as symlink attacks. The rule ensures developers explicitly check for the file or directory’s existence beforehand, preventing unintended permissions inheritance.
Resource Leaks
Availability is a critical aspect of security. After all, it’s the "A" in the good ol’ CIA triad (Confidentiality, Integrity, and Availability). Yet, it often doesn’t receive the attention it deserves in security discussions beyond the obvious Distributed Denial of Service (DDoS) attacks. Issues like goroutine leaks or unclosed resources can quietly degrade performance over time eventually leading to denial-of-service conditions if left unchecked.
Two rules in particular, missing-close-onfile and missing-defer-http, focus on common issues that are easy to overlook during reviews. These rules catch unclosed file descriptors and response bodies – issues that might seem minor but can have serious implications in long-running applications. For instance, forgetting to close these resources can lead to performance degradation and resource exhaustion.
The defer-in-loop rule detects the improper use of defer inside loops. In Go, calls to functions using defer within a loop aren’t executed at the end of each loop iteration. Instead, they are executed only when the enclosing function exits. This means that if, for example, you open a file within a loop and defer its closure in the same loop, the file descriptors won’t actually be released until the entire function completes. The rule helps prevent this situation, which could lead to performance degradation and resource exhaustion in your application.
Google Cloud Platform (GCP)-specific Rules
Our GCP-specific rules include creds-from-jwtconfig, which detects instances of Service Account Keys being loaded from JSON files. This rule was one of our earliest Semgrep additions to prevent developers from hardcoding sensitive credentials in code for talking with GCP resources.
We also developed gcs-path-traversal, which flags cases where user-provided filenames are used in GCS operations without proper sanitization. Unsanitized inputs could allow a user to manipulate file paths and access unintended files or directories within a bucket.
Optimizations
Under the optimizations category, the math-random-used rule is a refined version of an existing rule in the default Semgrep registry. The original rule flagged all uses of the math/rand library to ensure it wasn’t used for sensitive operations, like generating cryptographic keys. However, this approach generated excessive noise. Our modified rule excludes common safe use cases, such as using math/rand for generating random timeouts, reducing unnecessary alerts while maintaining security coverage.
Wrap-up
We hope these rules help improve the security of your Go applications. Stay tuned, as we plan to expand the repository with additional rules in the future. You can explore the rules here.
