0

I am seeking input on how to effectively utilize Python to commit and push a single file to an online GitHub repository.

The use case is pretty simple - I am pulling some data down from an API once per day, and I would like to send the pulled data to an online repo.

I am currently using pyGithub but I can only imagine that this is adding unnecessary complexity.

I am certain that my lack of success is due simply to a lack of experience. However, there does not appear to be any great tutorials on this topic (that I have found). The only solution I have come up with thus-far is indicated below (mostly swiped from another user here on Stack Overflow).

from github import Github
from github import InputGitTreeElement


def sendData(data):
    g = Github("-------------------------------------------")
    repo = g.get_user().get_repo("--------------")
    fileList = ["E:/----------------/test.txt"]
    fileName = "test.txt"
    commit_message = "test commit using Python"
    master_ref = repo.get_git_ref('heads/main')
    master_sha = master_ref.object.sha
    base_tree = repo.get_git_tree(master_sha)
    elementList = list()
    for file in fileList:
        with open(file) as input_file:
            data = input_file.read()
        element = InputGitTreeElement(fileName, '100644', 'blob', data) # https://docs.github.com/en/free-pro-team@latest/rest/reference/git#create-a-tree
        elementList.append(element)
    tree = repo.create_git_tree(elementList, base_tree)
    parent = repo.get_git_commit(master_sha)
    commit = repo.create_git_commit(commit_message, tree, [parent])
    master_ref.edit(commit.sha)

This solution does work to commit and push multiple or single files, so long as you have those files stored locally and know their file paths.

However, I am not sure how to use a file that is open in memory and doesn't have an existing file path, as currently required with the sendData() function.

Take the following function for example:

import requests
import json


def getData():
    apiKey = "-----------"
    try:
        response = requests.get('-----------%s' % apiKey)
        response.raise_for_status()
        # access JSON content
        jsonResponse = response.json()
        with open('data.json', 'w', encoding='utf-8') as data:
            json.dump(jsonResponse, data, ensure_ascii=False, indent=4)
        return data
    except requests.exceptions.HTTPError as http_err:
        print(f'HTTP error occurred: {http_err}')
    except Exception as err:
        print(f'Other error occurred: {err}')

The above works well to pull a JSON response from an API and return an open json file.

However, I need to now take this open json file and send it to my online GitHub repo. Is the answer to first save locally and then push the "saved file" to GitHub?

Ideally, I would like to simply pass the open JSON file to the sendData() function and have said function commit & push the file to my online GitHub repo.

I am hopeful that there is a more simple solution out there for committing/pushing a single file to a GitHub repo.

I am actively working on this project so this question will update often until a reasonable solution is found. If anyone has any thoughts or input, I would greatly appreciate it.

Current State

import requests
import json
from github import Github
from github import InputGitTreeElement


def getData():
    apiKey = "-----------------------------"
    try:
        response = requests.get('https://----------------.json?apiKey=%s' % apiKey)
        response.raise_for_status()
        # access JSON content
        jsonResponse = response.json()
        with open('data.json', 'w', encoding='utf-8') as data:
            json.dump(jsonResponse, data, ensure_ascii=False, indent=4)
        return data
    except requests.exceptions.HTTPError as http_err:
        print(f'HTTP error occurred: {http_err}')
    except Exception as err:
        print(f'Other error occurred: {err}')


def sendData(new_data):
    # GitHub access token - limited public repo access only
    g = Github("----------------------------------------")
    repo = g.get_user().get_repo("-----------")
    fileName = "test data.json"
    commit_message = "test commit using Python"
    master_ref = repo.get_git_ref('heads/main')
    master_sha = master_ref.object.sha
    base_tree = repo.get_git_tree(master_sha)
    elementList = list()
    element = InputGitTreeElement(fileName, '100644', 'blob', new_data)  # https://docs.github.com/en/free-pro-team@latest/rest/reference/git#create-a-tree
    elementList.append(element)
    tree = repo.create_git_tree(elementList, base_tree)
    parent = repo.get_git_commit(master_sha)
    commit = repo.create_git_commit(commit_message, tree, [parent])
    master_ref.edit(commit.sha)


def main():
    data = getData()
    sendData(data)

main()

Current Error
Traceback (most recent call last):
  File "<input>", line 45, in <module>
  File "<input>", line 43, in main
  File "<input>", line 33, in sendData
  File "C:\Users\chris\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\github\InputGitTreeElement.py", line 58, in __init__
    assert content is github.GithubObject.NotSet or isinstance(
AssertionError: <_io.TextIOWrapper name='data.json' mode='w' encoding='utf-8'>
4
  • Does this answer your question? How to push local files to github using python? (or post a commit via Python) Commented Nov 11, 2020 at 16:13
  • Could you elaborate on what you mean by 'constant issues'? Another example using subprocess here: steadylearner.com/blog/read/… Commented Nov 11, 2020 at 16:15
  • Hi Chris - welcome to SO! It might help you get more useful feedback if you could post some code examples of the things you've tried - both pyGithub and subprocess are modules you could use to accomplish what you're going for, so what you probably need is guidance on specific issues. Also - in git, "commit" and "push" are separate concepts / actions. Are you trying to do both via python? Or have you already made the commit (outside python), and only want to do the push in python? Commented Nov 11, 2020 at 16:26
  • I am also unsure of what the different parameters are for in the InputGitTreeElement class. The documentation from pyGithub is not entirely helpful. Commented Nov 11, 2020 at 17:00

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.