Tracking: On GHES, there is no fallback if this action is rate limited

[yacaovsnc]

Describe the bug
On GHES, we do not use the token input from user. So if this action is rate limited, there is no workaround.

Which version of the action are you using?

• v1
• v2
• Some other tag (such as v2.0.1 or master)

[killuazhu]

We run into the same issue... @yacaovsnc did you just ask your users to put a GH token in the action?

[dsame]

@yacaovsnc can you please describe your situation:

What is the reason it is not possible to use token input?
Which workaround do you prefer to have?

[scream314]

@dsame I guess the desired behaviour would be something like in case of https://github.com/actions/setup-node.

setup-node:

Run actions/setup-node@v3
Attempting to download 14...
API rate limit exceeded for 1.2.3.4. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Falling back to download directly from Node
Acquiring 14.20.0 - x64 from https://nodejs.org/dist/v14.20.0/node-v14.20.0-linux-x64.tar.gz
Extracting ...

(and then it continues after extracting)

setup-python:

Run actions/setup-python@v3
Version 3.x was not found in the local cache
Error: API rate limit exceeded for 1.2.3.4. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

(the action just stops after the rate limit error)

What is the reason it is not possible to use token input?

I'm not even sure what token should be used and where, many of the users do not have a github.com account, and in case of automated workflows there is no user involved at all.

Which workaround do you prefer to have?

Falling back to eg. python.com (the same way as setup-node falls back to nodejs.org.

[aboutthatjazz]

A friendly bump as this issue if forcing customers to workaround this issue in a manner that increases the complexity to provide support for our customers.

[zetaab]

latest GHES https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#using-setup-python-on-ghes instruction says that we should define github.com token in case of rate limits. However, that does not work either.

workflow:

    - name: Caching Python 3.8
      uses: actions/setup-python@v4
      with:
        token: ${{ secrets.GH_DOTCOM_TOKEN }}
        python-version: 3.8

logs:

##[debug]Evaluating condition for step: 'Caching Python 3.8'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Caching Python 3.8
##[debug]Register post job cleanup for action: actions/setup-python@v4
##[debug]Loading inputs
##[debug]Evaluating: secrets.GH_DOTCOM_TOKEN
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GH_DOTCOM_TOKEN'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Loading env
Run actions/setup-python@v4
##[debug]Python is expected to be installed into /opt/hostedtoolcache
##[debug]Semantic version spec of 3.8 is 3.8
##[debug]isExplicit: 
##[debug]explicit? false
##[debug]evaluating 0 versions
##[debug]match not found
Version 3.8 was not found in the local cache
##[debug]Getting manifest from actions/python-versions@main
Error: API rate limit exceeded for x.x.x.x. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
##[debug]Node Action run completed with exit code 1
##[debug]Finishing: Caching Python 3.8

btw is there reason

setup-python/src/install-python.ts

Line 13 in 98c991d

export const MANIFEST_URL = `https://raw.githubusercontent.com/${MANIFEST_REPO_OWNER}/${MANIFEST_REPO_NAME}/${MANIFEST_REPO_BRANCH}/versions-manifest.json`;

why this is hardcoded to github.com? Why people could not sync python-versions to GHES and use that from local installation?

[dsame]

Hello, @zetaab

However, that does not work either.

Can you please double check you've set up the secret GH_DOTCOM_TOKEN for the repository setting? Pay attentions - you used namely repository setting, not organisation and not user profile setting.

I suspect something wrong with this because i do not see in the log you've sent a line ##[debug]=> set auth as it would be expected in case if the secret had set https://github.com/actions/toolkit/blob/192c26f865db0bb700aa0a40d193d08b57123949/packages/tool-cache/src/tool-cache.ts#L97.

[dsame]

btw is there reason

setup-python/src/install-python.ts

Line 13 in 98c991d

export const MANIFEST_URL = `https://raw.githubusercontent.com/${MANIFEST_REPO_OWNER}/${MANIFEST_REPO_NAME}/${MANIFEST_REPO_BRANCH}/versions-manifest.json`;

why this is hardcoded to github.com? Why people could not sync python-versions to GHES and use that from local installation?

@zetaab , i investigated the code you've pointed out, but honestly, i can not see the connection between this MANIFEST_URL const and the problem. The exact root of the exception is here https://github.com/actions/toolkit/blob/192c26f865db0bb700aa0a40d193d08b57123949/packages/tool-cache/src/tool-cache.ts#L605 and it uses API endpoint URL https://github.com/actions/toolkit/blob/192c26f865db0bb700aa0a40d193d08b57123949/packages/tool-cache/src/tool-cache.ts#L596

This why i tend to think you've been mislead by the documentation and assumed GH_DOTCOM_TOKEN was predifined.

This doc page https://docs.github.com/en/actions/security-guides/encrypted-secrets describes how to set the secrets.

[zetaab]

@dsame I am not assuming that GH_DOTCOM_TOKEN was predefined. I have defined that as Github actions secret and the value is PAT from github.com. Still the solution does not work.

screenshot from same repo where the logs were found

[dsame]

Hello @zetaab i double checked the possible problems with the pipeline and still suspect the secret with the token is not set.

This is the sample build and there's a line in the log "set auth" confirming the token is set:

##[debug]Getting manifest from actions/python-versions@main
##[debug]set auth

by contrast the log you've pasted does not have such linek

##[debug]Getting manifest from actions/python-versions@main

indicating the condition

  if (auth) {
    core.debug('set auth')
    headers.authorization = auth
  }

is evaluated to false. From the other side i see the secret is set

##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GH_DOTCOM_TOKEN'
##[debug]=> '***'
##[debug]Result: '***'

Thus, there might be only 2 reason for token to do not work: if there's misspelling or the secret is empty.

In order to check the contenten of the secret please use the snippet below

     run: |
       echo ${token} > /tmp/token
       ls -l /tmp/token
     with:
       token: ${{ secrets.GH_DOTCOM_TOKEN }}

You should see a file with the length greater than zero. In case if the secrete is set and there's no misspelling i have to ask you to create a demo repo with the exact pipeline that does no accept the token in order i become able to fork it and investigate.

[zetaab]

@dsame here is our workflow file https://gist.github.com/zetaab/2c877a061c5f71deeafc1779e6508827 (I removed some docker registry things, but you will hopefully see the main point in this). So we are trying to build github actions runner image which is containing cached tools. However, the problem is that our Kubernetes cluster is quite busy building other people stuff so we are hitting rate limits all the time. Our cluster is something like 1TB of memory and tens of nodes itself. Everything has same egress ip address towards internet.

output from the step that you were asking to add https://gist.github.com/zetaab/da5e1fba1688c2e1d8376da237b6a088 as you can see the token is working and it has permission repo in github.

However, when it starts to execute python 3.8 step it will fail. https://gist.github.com/zetaab/44af24da1fdc9bd8128037a418eff1d0

[zetaab]

I think one problem could be now the format how the secret is inserted. For me I am just copy pasting the PAT itself.

If we check for instance setup-java its just passing token as is to header https://github.com/actions/setup-java/blob/a18c333f3f14249953dab3e186e5e21bf3390f1d/src/distributions/microsoft/installer.ts#L74 and https://github.com/actions/setup-java/blob/a18c333f3f14249953dab3e186e5e21bf3390f1d/src/distributions/microsoft/installer.ts#L84

However, setup-python is always adding prefix token to it https://github.com/actions/setup-python/blob/main/src/install-python.ts#L9

what is the correct way?

% curl -s -H 'Authorization: <token>' https://api.github.com -I|grep ratelimit
x-ratelimit-limit: 60
x-ratelimit-remaining: 56
x-ratelimit-reset: 1664976862
x-ratelimit-used: 4
x-ratelimit-resource: core

% curl -s -H 'Authorization: token <token>' https://api.github.com -I|grep ratelimit
x-ratelimit-limit: 5000
x-ratelimit-remaining: 4995
x-ratelimit-reset: 1664975450
x-ratelimit-used: 5
x-ratelimit-resource: core

% curl -s -H 'Authorization: Bearer <token>' https://api.github.com -I|grep ratelimit
x-ratelimit-limit: 5000
x-ratelimit-remaining: 4994
x-ratelimit-reset: 1664975450
x-ratelimit-used: 6
x-ratelimit-resource: core

seems that if I just copy paste token at least setup-java is not working in similar way than setup-python. However, setup-python should work correctly if I am reading this code correctly (its adding token prefix)

[zetaab]

@dsame I found the issue. The issue is that we are using setup-python@v4. However, the v4 does not contain latest fixes from the main. This commit 98c991d needs to be tagged and after that everything is working fine. If I use setup-python@main it starts to work!

So could someone move v4 tag forward and make 4.3.0?