[Go] Add Unicode Bypass Validation query, test and help file

[Sim4n6]

This pull request adds a Unicode Bypass Validation (UBV) query, tests, and the help file.

The UBV query checks for a Post-Unicode Normalization in a Golang codebase that leads to some security issues such as the bypasses of a String validation, regex verification, and escape functions.

[github-actions]

QHelp previews:

go/ql/src/experimental/CWE-176/UnicodeBypassValidation.qhelp

Bypass Logical Validation Using Unicode Characters

Security checks bypass due to a Unicode transformation

If ever a unicode tranformation is performed after some security checks or logical validation, the latter could be bypassed due to a potential Unicode characters collision. The validation of concern are any character escaping, any regex validation or any string verification.

Recommendation

Perform a Unicode normalization before the logical validation.

Example

The following example showcases the bypass of all checks performed by html.EscapeString() due to a post-unicode normalization.

For instance: the character U+FE64 (﹤) is not filtered-out by the flask escape function. But due to the Unicode normalization, the character is transformed and would become U+003C (< ).

package main

import (
	"fmt"
	"html"
	"net/http"

	"golang.org/x/text/unicode/norm"
)

func main() {}

func bad() {
	http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {

		unicode_input := req.URL.Query().Get("unicode_input")
		escaped := html.EscapeString(unicode_input)
		unicode_norm := norm.NFKC.String(escaped)
		fmt.Println(w, "Results: %q", unicode_norm)
	})
}

References

• Research study: Unicode vulnerabilities that could bYte you and Unicode pentest cheatsheet.
• Common Weakness Enumeration: CWE-176.

[Sim4n6]

Sorry about the delay, I will be working on this query this weekend.

[smowton]

Mostly looks good, couple more adjustments

[Sim4n6]

A regex match function calls could be used as a barrier guard, commit a64a998.

[smowton]

This is looking good now! I note the bounty application wants you to submit a Golang CVE -- once that's done I think securitylab will now proceed to check his query's accuracy.

[Sim4n6]

Thanks, no problem @smowton