We get this error when uploading a large file (more than 10Mb but less than 100Mb):

403 POST https://www.googleapis.com/upload/storage/v1/b/dm-scrapes/o?uploadType=resumable: ('Response headers must contain header', 'location')

Or this error when the file is more than 5Mb

403 POST https://www.googleapis.com/upload/storage/v1/b/dm-scrapes/o?uploadType=multipart: ('Request failed with status code', 403, 'Expected one of', <HTTPStatus.OK: 200>)

It seems that this API is looking at the file size and trying to upload it via multi part or resumable method. I can't imagine that is something that as a caller of this API I should be concerned with. Is the problem somehow related to permissions? Does the bucket need special permission do it can accept multipart or resumable upload.

from google.cloud import storage

try:
    client = storage.Client()
    bucket = client.get_bucket('my-bucket')
    blob = bucket.blob('blob-name')
    blob.upload_from_filename(zip_path, content_type='application/gzip')

except Exception as e:
    print(f'Error in uploading {zip_path}')
    print(e)

We run this inside a Kubernetes pod so the permissions get picked up by storage.Client() call automatically.

We already tried these:

• Can't upload with gsutil because the container is Python 3 and gsutil does not run in python 3.

• Tried this example: but runs into the same error: ('Response headers must contain header', 'location')

• There is also this library. But it is basically alpha quality with little activity and no commits for a year.

• Upgraded to google-cloud-storage==1.13.0

Thanks in advance