404 Client Error: Not Found for url

[t3chn0m4g3]

When running GitHub-Watchman against https://github.com with a personalized access token as described in the README I am only getting the following errors.

Tried this with two different accounts and different repos, but the error stays the same. I also widened the auth scope of the token to all scopes which made no difference.

Maybe the API changed on GitHub, a query for password with user was successful (at least with curl, https://github.com/search?q=user%3Amyusername+password).

{"localtime": "2021-01-15 17:36:13,456", "level": "INFO", "source": "GitHub Watchman", "message": "GitHub Watchman started execution"}
{"localtime": "2021-01-15 17:36:13,456", "level": "INFO", "source": "GitHub Watchman", "message": "Version: 1.0.1"}
{"localtime": "2021-01-15 17:36:13,456", "level": "INFO", "source": "GitHub Watchman", "message": "Importing rules..."}
{"localtime": "2021-01-15 17:36:13,504", "level": "INFO", "source": "GitHub Watchman", "message": "16 rules loaded"}
{"localtime": "2021-01-15 17:36:13,504", "level": "INFO", "source": "GitHub Watchman", "message": "Getting everything..."}
{"localtime": "2021-01-15 17:36:13,504", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack Webhooks in code"}
{"localtime": "2021-01-15 17:36:13,692", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=https%3A%2F%2Fhooks.slack.com%2F&page=1"}
{"localtime": "2021-01-15 17:36:13,693", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack Webhooks in commits"}
{"localtime": "2021-01-15 17:36:13,819", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=https%3A%2F%2Fhooks.slack.com%2F&page=1"}
{"localtime": "2021-01-15 17:36:13,819", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack Webhooks in issues"}
{"localtime": "2021-01-15 17:36:13,950", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=https%3A%2F%2Fhooks.slack.com%2F&page=1"}
{"localtime": "2021-01-15 17:36:13,950", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack Webhooks in repositories"}
{"localtime": "2021-01-15 17:36:14,070", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=https%3A%2F%2Fhooks.slack.com%2F&page=1"}
{"localtime": "2021-01-15 17:36:14,070", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Miscelaneous Private Keys in code"}
{"localtime": "2021-01-15 17:36:14,186", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=%22BEGIN+DSA+PRIVATE%22&page=1"}
{"localtime": "2021-01-15 17:36:14,187", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Miscelaneous Private Keys in commits"}
{"localtime": "2021-01-15 17:36:14,307", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=%22BEGIN+DSA+PRIVATE%22&page=1"}
{"localtime": "2021-01-15 17:36:14,307", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Miscelaneous Private Keys in issues"}
{"localtime": "2021-01-15 17:36:14,429", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=%22BEGIN+DSA+PRIVATE%22&page=1"}
{"localtime": "2021-01-15 17:36:14,429", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Miscelaneous Private Keys in repositories"}
{"localtime": "2021-01-15 17:36:14,548", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=%22BEGIN+DSA+PRIVATE%22&page=1"}
{"localtime": "2021-01-15 17:36:14,548", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Azure Service Account Files in code"}
{"localtime": "2021-01-15 17:36:14,665", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=azure+extension%3Ajson&page=1"}
{"localtime": "2021-01-15 17:36:14,665", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Client Secrets in code"}
{"localtime": "2021-01-15 17:36:14,781", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=%22client_secret%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:14,781", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Client Secrets in commits"}
{"localtime": "2021-01-15 17:36:14,906", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=%22client_secret%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:14,906", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Client Secrets in issues"}
{"localtime": "2021-01-15 17:36:15,028", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=%22client_secret%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:15,028", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Client Secrets in repositories"}
{"localtime": "2021-01-15 17:36:15,150", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=%22client_secret%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:15,151", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for PGP Private Keys in code"}
{"localtime": "2021-01-15 17:36:15,268", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=%22BEGIN+PGP+PRIVATE+KEY+BLOCK%22&page=1"}
{"localtime": "2021-01-15 17:36:15,268", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for PGP Private Keys in commits"}
{"localtime": "2021-01-15 17:36:15,384", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=%22BEGIN+PGP+PRIVATE+KEY+BLOCK%22&page=1"}
{"localtime": "2021-01-15 17:36:15,384", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for PGP Private Keys in issues"}
{"localtime": "2021-01-15 17:36:15,500", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=%22BEGIN+PGP+PRIVATE+KEY+BLOCK%22&page=1"}
{"localtime": "2021-01-15 17:36:15,500", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for PGP Private Keys in repositories"}
{"localtime": "2021-01-15 17:36:15,634", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=%22BEGIN+PGP+PRIVATE+KEY+BLOCK%22&page=1"}
{"localtime": "2021-01-15 17:36:15,634", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Access Tokens in code"}
{"localtime": "2021-01-15 17:36:15,754", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=%22access_token%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:15,754", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Access Tokens in commits"}
{"localtime": "2021-01-15 17:36:15,871", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=%22access_token%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:15,871", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Access Tokens in issues"}
{"localtime": "2021-01-15 17:36:16,000", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=%22access_token%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:16,001", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Access Tokens in repositories"}
{"localtime": "2021-01-15 17:36:16,122", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=%22access_token%3A%22&page=1"}
{"localtime": "2021-01-15 17:36:16,122", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Private Tokens in code"}
{"localtime": "2021-01-15 17:36:16,251", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=%22%27PRIVATE-TOKEN%27%3A+%22&page=1"}
{"localtime": "2021-01-15 17:36:16,251", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Private Tokens in commits"}
{"localtime": "2021-01-15 17:36:16,372", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=%22%27PRIVATE-TOKEN%27%3A+%22&page=1"}
{"localtime": "2021-01-15 17:36:16,373", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Private Tokens in issues"}
{"localtime": "2021-01-15 17:36:16,495", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=%22%27PRIVATE-TOKEN%27%3A+%22&page=1"}
{"localtime": "2021-01-15 17:36:16,496", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Private Tokens in repositories"}
{"localtime": "2021-01-15 17:36:16,627", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=%22%27PRIVATE-TOKEN%27%3A+%22&page=1"}
{"localtime": "2021-01-15 17:36:16,627", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Passwords in code"}
{"localtime": "2021-01-15 17:36:16,751", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=password%3A&page=1"}
{"localtime": "2021-01-15 17:36:16,751", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Passwords in commits"}
{"localtime": "2021-01-15 17:36:16,869", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=password%3A&page=1"}
{"localtime": "2021-01-15 17:36:16,869", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Passwords in issues"}
{"localtime": "2021-01-15 17:36:17,015", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=password%3A&page=1"}
{"localtime": "2021-01-15 17:36:17,015", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Passwords in repositories"}
{"localtime": "2021-01-15 17:36:17,156", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=password%3A&page=1"}
{"localtime": "2021-01-15 17:36:17,156", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Azure API Tokens in code"}
{"localtime": "2021-01-15 17:36:17,276", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=login.microsoftonline.com&page=1"}
{"localtime": "2021-01-15 17:36:17,277", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Azure API Tokens in commits"}
{"localtime": "2021-01-15 17:36:17,403", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=login.microsoftonline.com&page=1"}
{"localtime": "2021-01-15 17:36:17,404", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Azure API Tokens in issues"}
{"localtime": "2021-01-15 17:36:17,523", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=login.microsoftonline.com&page=1"}
{"localtime": "2021-01-15 17:36:17,523", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Azure API Tokens in repositories"}
{"localtime": "2021-01-15 17:36:17,943", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=login.microsoftonline.com&page=1"}
{"localtime": "2021-01-15 17:36:17,943", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for S3 Configuration Files in code"}
{"localtime": "2021-01-15 17:36:18,062", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=secret+in%3Afile+extension%3As3cfg&page=1"}
{"localtime": "2021-01-15 17:36:18,063", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack API Tokens in code"}
{"localtime": "2021-01-15 17:36:18,183", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=xoxb&page=1"}
{"localtime": "2021-01-15 17:36:18,183", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack API Tokens in commits"}
{"localtime": "2021-01-15 17:36:18,305", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=xoxb&page=1"}
{"localtime": "2021-01-15 17:36:18,305", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack API Tokens in issues"}
{"localtime": "2021-01-15 17:36:18,422", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=xoxb&page=1"}
{"localtime": "2021-01-15 17:36:18,422", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Slack API Tokens in repositories"}
{"localtime": "2021-01-15 17:36:18,545", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=xoxb&page=1"}
{"localtime": "2021-01-15 17:36:18,546", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Interesting Potentially Sensitive Files in code"}
{"localtime": "2021-01-15 17:36:18,663", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=BEGIN+CERTIFICATE+in%3Afile+extension%3Aopenvpn&page=1"}
{"localtime": "2021-01-15 17:36:18,664", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Bearer Tokens in code"}
{"localtime": "2021-01-15 17:36:18,789", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=Bearer&page=1"}
{"localtime": "2021-01-15 17:36:18,790", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Bearer Tokens in commits"}
{"localtime": "2021-01-15 17:36:18,914", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=Bearer&page=1"}
{"localtime": "2021-01-15 17:36:18,915", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Bearer Tokens in issues"}
{"localtime": "2021-01-15 17:36:19,033", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=Bearer&page=1"}
{"localtime": "2021-01-15 17:36:19,033", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Bearer Tokens in repositories"}
{"localtime": "2021-01-15 17:36:19,155", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=Bearer&page=1"}
{"localtime": "2021-01-15 17:36:19,155", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for AWS API Tokens in code"}
{"localtime": "2021-01-15 17:36:19,271", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=ASIA&page=1"}
{"localtime": "2021-01-15 17:36:19,271", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for AWS API Tokens in commits"}
{"localtime": "2021-01-15 17:36:19,395", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=ASIA&page=1"}
{"localtime": "2021-01-15 17:36:19,396", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for AWS API Tokens in issues"}
{"localtime": "2021-01-15 17:36:19,513", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=ASIA&page=1"}
{"localtime": "2021-01-15 17:36:19,513", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for AWS API Tokens in repositories"}
{"localtime": "2021-01-15 17:36:19,631", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=ASIA&page=1"}
{"localtime": "2021-01-15 17:36:19,632", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Google API Tokens in code"}
{"localtime": "2021-01-15 17:36:19,753", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=AIza&page=1"}
{"localtime": "2021-01-15 17:36:19,753", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Google API Tokens in commits"}
{"localtime": "2021-01-15 17:36:19,866", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/commits?per_page=100&q=AIza&page=1"}
{"localtime": "2021-01-15 17:36:19,867", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Google API Tokens in issues"}
{"localtime": "2021-01-15 17:36:19,983", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/issues?per_page=100&q=AIza&page=1"}
{"localtime": "2021-01-15 17:36:19,983", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for Google API Tokens in repositories"}
{"localtime": "2021-01-15 17:36:20,132", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/repositories?per_page=100&q=AIza&page=1"}
{"localtime": "2021-01-15 17:36:20,132", "level": "INFO", "source": "GitHub Watchman", "message": "Searching for GCP Service Account Files in code"}
{"localtime": "2021-01-15 17:36:20,249", "level": "CRITICAL", "source": "GitHub Watchman", "message": "404 Client Error: Not Found for url: https://github.com/api/v3/search/code?per_page=100&q=private_key+in%3Afile+extension%3Ajson&page=1"}
{"localtime": "2021-01-15 17:36:20,249", "level": "INFO", "source": "GitHub Watchman", "message": "++++++Audit completed++++++"}

[ankitdevnalkar]

facing the same issue.

Any workaround?

[ankitdevnalkar]

I have fixed 403 rate limiting and a few minor issues and made a workaround to run watchman for a particular organization.
you can use the following release

https://github.com/ankitdevnalkar/github-watchman

[t3chn0m4g3]

@ankitdevnalkar Thanks!

[PaperMtn]

@ankitdevnalkar Great! I haven't had time to look at this myself (basically all of this year). Feel free to create a pull request and I can merge your improvements in

[ankitdevnalkar]

@PaperMtn proposed a change to fix rate limiting issue.
Also, below is the workaround to run watchman for a particular organization. You can simply put this in the README file.

Run Watchman For Perticular Organisation

• After installing github-watchman in system, open github_wrapper.py located at /home/<YourLocalUser>/.local/lib/python3.6/site-packages/github_watchman/github_wrapper.py
• Find params = {(should be around line 82)
• Set query parameter to 'q':'org:<YourOrgName> '+query and save file.

hope this helps 🙂 !