GitHub Connector Tools

​

Overview

​

Tool Name

github_connector_tools

​

Purpose

​

Functions Available

1. git_action: Core Git operations including repository management, branching, committing, and history tracking.
2. file: File operations for reading, writing, searching, and managing repository content.
3. git_action (GitHub-specific actions): GitHub platform features including pull requests, remote repository management, and credential storage.

​

Key Features

​

Input Parameters for Each Function

​

git_action

​

file

​

Use Cases

1. Automated Documentation Generate and commit data dictionaries, analysis reports, or pipeline documentation directly to GitHub repositories.
2. Code Generation & Versioning Create dbt models, SQL scripts, or Python notebooks and version them in GitHub for team collaboration.
3. Data Pipeline Deployment Clone configuration repositories, update parameters, commit changes, and create pull requests for review.
4. Collaborative Data Projects Share analysis notebooks, datasets, and findings with team members through structured GitHub workflows.
5. Backup & Disaster Recovery Automatically commit critical configurations, metadata, and artifacts to GitHub for version-controlled backups.

​

Workflow/How It Works

1. Step 1: Authenticate with GitHub Store GitHub credentials securely using Personal Access Token:
   Copy

   Ask AI

   git_action(
       action="store_github_credentials",
       username="your_github_username",
       token="ghp_your_personal_access_token"
   )
   

2. Step 2: Create or Clone Repository Start with a new repository or clone an existing one:
   Copy

   Ask AI

   # Create new repository
   git_action(
       action="create_repo",
       repo_id="my_data_project"
   )
   
   # Or clone from GitHub
   git_action(
       action="clone_repo",
       repo_id="analytics_pipeline",
       url="https://github.com/organization/analytics-pipeline.git"
   )
   

3. Step 3: Manage Files Create, read, and organize files within the repository:
   Copy

   Ask AI

   # Write documentation
   file(
       action="write",
       repo_id="my_data_project",
       file_path="docs/data_dictionary.md",
       content="# Data Dictionary\n\n## Customer Table\n..."
   )
   
   # Search for files
   file(
       action="find",
       repo_id="my_data_project",
       pattern="*.sql"
   )
   

4. Step 4: Commit Changes Track changes with descriptive commit messages:
   Copy

   Ask AI

   git_action(
       action="commit",
       repo_id="my_data_project",
       commit_message="Add customer data dictionary and initial ETL scripts"
   )
   

5. Step 5: Create Pull Request Collaborate with team through pull requests:
   Copy

   Ask AI

   git_action(
       action="create_pull_request",
       repo_id="my_data_project",
       branch_name="feature/new-pipeline",
       title="Add new customer segmentation pipeline",
       body="This PR introduces customer segmentation logic with dbt models."
   )
   

6. Step 6: Push to GitHub Sync local changes to remote repository:
   Copy

   Ask AI

   git_action(
       action="push",
       repo_id="my_data_project"
   )
   

​

Integration Relevance

• project_manager_tools to track data projects and link GitHub repositories to missions.
• data_connector_tools to export analysis results and commit them to version control.
• dbt_action to manage dbt projects in GitHub with full version control.
• file_manager_tools to organize artifacts before committing to repositories.
• slack_tools to notify teams when pull requests are created or commits are pushed.

​

Configuration Details

• Personal Access Token: Generate from GitHub Settings → Developer settings → Personal access tokens. Required scopes: repo, workflow.
• Repository Naming: Use lowercase with hyphens (e.g., data-pipeline-prod) for consistency.
• Branch Strategy: Use feature branches (feature/, bugfix/) for changes; protect main branch.
• Commit Messages: Follow conventional commits format: feat:, fix:, docs:, refactor:.
• File Paths: Always use forward slashes (/) and relative paths without leading slash.
• Large Files: GitHub has 100MB file size limit; use Git LFS for larger artifacts or store in data platforms.

​

Limitations or Notes

1. File Size Limits: GitHub limits individual files to 100MB; repositories to 1GB recommended size.
2. API Rate Limits: GitHub API has rate limits (5,000 requests/hour for authenticated users); batch operations when possible.
3. Private Repository Access: Requires appropriate permissions and PAT scopes for private repos.
4. Binary Files: Git is optimized for text files; large binary files can slow repository performance.
5. Merge Conflicts: Automatic merging may fail with conflicts; manual resolution required.
6. Branch Protection: Protected branches may prevent direct pushes; use pull requests instead.
7. Network Dependency: All operations require internet connectivity to GitHub servers.

​

Supported Actions

​

Not Supported

​

Output

• clone_repo: Confirmation with repository path and clone details.
• commit: Commit hash, files changed, and commit message confirmation.
• push: Push status, branch name, and remote URL.
• get_history: List of commits with hash, author, date, and message.
• get_status: Current branch, staged/unstaged files, and sync status.
• create_pull_request: PR number, URL, and creation confirmation.
• file operations: Success confirmation, file paths, and content previews.
• list_repos: Table of repositories with names, paths, and status.
• Errors: Detailed error messages with resolution guidance (e.g., authentication failures, merge conflicts).

​

Best Practices

​

Example: Complete Data Project Workflow

# 1. Store GitHub credentials (one-time setup)
git_action(
    action="store_github_credentials",
    username="data_engineer",
    token="ghp_abc123..."
)

# 2. Clone existing project repository
git_action(
    action="clone_repo",
    repo_id="customer_analytics",
    url="https://github.com/company/customer-analytics.git"
)

# 3. Create feature branch for new analysis
git_action(
    action="create_branch",
    repo_id="customer_analytics",
    branch_name="feature/churn-analysis"
)

git_action(
    action="switch_branch",
    repo_id="customer_analytics",
    branch_name="feature/churn-analysis"
)

# 4. Create analysis notebook
file(
    action="write",
    repo_id="customer_analytics",
    file_path="notebooks/churn_analysis.py",
    content="""
# Customer Churn Analysis
import pandas as pd

# Load data
customers = pd.read_csv('data/customers.csv')

# Analysis logic...
"""
)

# 5. Create documentation
file(
    action="write",
    repo_id="customer_analytics",
    file_path="docs/churn_methodology.md",
    content="# Churn Analysis Methodology\n\n## Overview\n..."
)

# 6. Commit changes
git_action(
    action="commit",
    repo_id="customer_analytics",
    commit_message="feat: Add customer churn analysis with documentation"
)

# 7. Push to GitHub
git_action(
    action="push",
    repo_id="customer_analytics",
    branch_name="feature/churn-analysis"
)

# 8. Create pull request for review
git_action(
    action="create_pull_request",
    repo_id="customer_analytics",
    branch_name="feature/churn-analysis",
    title="Add Customer Churn Analysis",
    body="This PR adds:\n- Churn prediction model\n- Analysis notebooks\n- Documentation"
)

# 9. Check repository status
git_action(
    action="get_status",
    repo_id="customer_analytics"
)

​

Advanced Features

​

Bulk File Operations

file(
    action="copy",
    repo_id="customer_analytics",
    source_patterns=["*.sql", "*.py"],
    target_path="archive/",
    preserve_structure=True
)

​

Search Across Files

file(
    action="search",
    repo_id="customer_analytics",
    pattern="SELECT.*FROM customers",
    regex=True
)

​

Fetch Deep History

git_action(
    action="fetch_more_history",
    repo_id="customer_analytics",
    depth=100
)

git_action(
    action="get_history",
    repo_id="customer_analytics",
    max_count=50
)

​

Troubleshooting