GitLab Connector Tools

​

Overview

​

Tool Name

git_and_file_tools

​

Purpose

​

Functions Available

1. git_action: Core Git operations including repository management, branching, committing, and history tracking.
2. file: File operations for reading, writing, searching, and managing repository content.
3. git_action (GitLab-specific actions): GitLab platform features including merge requests, remote repository management, and credential storage.

​

Key Features

​

Input Parameters for Each Function

​

git_action

​

file

​

Use Cases

1. Automated Documentation Generate and commit data dictionaries, API documentation, or pipeline specifications directly to GitLab repositories with automatic CI/CD triggers.
2. Code Generation & Versioning Create dbt models, SQL scripts, Terraform configurations, or Python notebooks and version them in GitLab for team collaboration.
3. CI/CD Pipeline Integration Update configuration files, trigger pipeline runs, and track deployment artifacts through GitLab’s integrated DevOps platform.
4. Collaborative Data Projects Share analysis notebooks, datasets, experiment results, and findings with team members through structured GitLab workflows.
5. Enterprise Data Governance Maintain audit trails, approval workflows, and compliance documentation with GitLab’s advanced merge request and access control features.

​

Workflow/How It Works

1. Step 1: Authenticate with GitLab Store GitLab credentials securely using Personal Access Token:
   git_action(
       action="store_gitlab_credentials",
       gitlab_token="YOUR_GITLAB_PAT"
   )
   
2. Step 2: Create or Clone Repository Start with a new repository or clone an existing one:
   # Create new repository
   git_action(
       action="create_repo",
       repo_id="my_data_project"
   )
   
   # Or clone from GitLab
   git_action(
       action="clone_repo",
       repo_id="analytics_pipeline",
       url="https://gitlab.com/organization/analytics-pipeline.git"
   )
   

3. Step 3: Manage Files Create, read, and organize files within the repository:
   # Write documentation
   file(
       action="write",
       repo_id="my_data_project",
       file_path="docs/data_dictionary.md",
       content="# Data Dictionary\n\n## Customer Table\n..."
   )
   
   # Search for configuration files
   file(
       action="find",
       repo_id="my_data_project",
       pattern="*.yml"
   )
   
4. Step 4: Commit Changes Track changes with descriptive commit messages:
   git_action(
       action="commit",
       repo_id="my_data_project",
       commit_message="docs: Add customer data dictionary and ETL specifications"
   )
   

5. Step 5: Create Merge Request Collaborate with team through merge requests:
   git_action(
       action="create_pull_request",  # Same action works for GitLab MRs
       repo_id="my_data_project",
       branch_name="feature/new-pipeline",
       title="Add new customer segmentation pipeline",
       body="This MR introduces customer segmentation logic with dbt models and CI/CD integration."
   )
   
6. Step 6: Push to GitLab Sync local changes to remote repository:
   git_action(
       action="push",
       repo_id="my_data_project"
   )
   

​

Managed GitLab API Commits

git_action(
    action="commit_files_via_api",
    repo_id="customer_analytics",
    branch_name="wip/alice/churn-docs",
    start_branch="main",
    expected_branch_head_sha="abc123def4567890",
    commit_message="docs: refresh churn methodology",
    files=[
        {
            "path": "docs/churn_model_methodology.md",
            "source_path": "/workspace/customer_analytics/docs/churn_model_methodology.md",
        }
    ],
)

​

Integration Relevance

• project_manager_tools to track data projects and link GitLab repositories to missions.
• data_connector_tools to export analysis results and commit them to version control.
• dbt_action to manage dbt projects in GitLab with full version control and CI/CD.
• file_manager_tools to organize artifacts before committing to repositories.
• slack_tools to notify teams when merge requests are created or commits are pushed.
• scheduler_tools to automate regular commits, syncs, or pipeline triggers.

​

Configuration Details

• Personal Access Token: Generate from GitLab Settings → Access Tokens. Required scopes: api, read_repository, write_repository. Include read_api when you need list_remote_repos (and Context Graph Select repos) to list group/organization projects, not only personal ones.
• Git Commit Author (git_author_mode): Optional select field on the saved GitLab connection. Default: genesis_bot. Supported values: genesis_bot, authenticated_git_user, custom.
• Git Author Name (git_author_name) and Git Author Email (git_author_email): Optional string fields on the saved GitLab connection. Genesis shows them only when git_author_mode="custom", and both are required together in that mode.
• Repository Naming: Use lowercase with hyphens (e.g., data-pipeline-prod) for consistency.
• Branch Strategy: Use feature branches (feature/, bugfix/, hotfix/) for changes; protect main and develop branches.
• Commit Messages: Follow conventional commits format: feat:, fix:, docs:, ci:, refactor:.
• File Paths: Always use forward slashes (/) and relative paths without leading slash.
• Large Files: GitLab has 10GB repository size limit; use Git LFS for files over 100MB.
• CI/CD Integration: Merge requests can automatically trigger GitLab CI/CD pipelines defined in .gitlab-ci.yml.

​

Limitations or Notes

1. Repository Size: GitLab recommends keeping repositories under 10GB; large repos may have performance issues.
2. API Rate Limits: GitLab has rate limits (varies by tier—Free: 300/min, Premium: 1000/min); batch operations when possible.
3. Private Repository Access: Requires appropriate permissions and PAT scopes for private/internal repos.
4. Binary Files: Git is optimized for text files; large binary files can slow repository performance.
5. Merge Conflicts: Automatic merging may fail with conflicts; manual resolution required.
6. Branch Protection: Protected branches prevent direct pushes; use merge requests instead.
7. Network Dependency: All operations require internet connectivity to GitLab servers.
8. GitLab Version: Some features may require specific GitLab versions (especially self-hosted instances).

​

Supported Actions

​

Not Supported

​

Output

• clone_repo: Confirmation with repository path and clone details.
• commit: Commit hash, files changed, and commit message confirmation.
• push: Push status, branch name, and remote URL.
• get_history: List of commits with hash, author, date, and message.
• get_status: Current branch, staged/unstaged files, and sync status.
• create_pull_request: MR number, URL (IID), and creation confirmation.
• get_pull_request_info: MR details including status, approvals, pipeline status, and discussion threads.
• file operations: Success confirmation, file paths, content previews, batch read results with per-file success or error details plus content and size for successful reads. Batch reads can also surface per-file truncated, truncation_info, continuation, encoding, file_type, total_size, returned_size, returned_lines, total_lines, start_line, end_line, start_byte, and end_byte, plus top-level warnings, files_truncated, files_skipped, skipped_preview, and continuation_hint when the response is cut short.
• Large path and archive previews: file(action="list") and file(action="find") can return files_preview, files_truncated, files_omitted_count, total_files, files_continuation_hint, note, and narrowing_hint when many paths match. file(action="unzip") can similarly preview extracted_files and download_links with fields such as extracted_files_preview, extracted_files_truncated, extracted_files_omitted_count, total_extracted_files, extracted_files_continuation_hint, download_links_preview, download_links_truncated, download_links_omitted_count, total_download_links, and download_links_continuation_hint.
• Zero-result repo search warning: git_action(action="search"), git_action(action="grep"), file(action="search"), and file(action="fast_search") can include _branch_warning when a repo on a non-default branch returns 0 matches. The object includes current_branch, default_branch, and message, which directs you to switch to the default branch, run git_action(action="pull"), and retry the search.
• list_repos: Repository ids by default. With details=true, each detail row includes mode, has_git_dir, has_remote, scope, owner_user_id, owner_bot_id, and undeletable, plus credentials-redacted remote_url when present, clone_source when Genesis can infer the remote platform, and path when include_paths=true.
• list_remote_repos: personal_repos (flat list of user-namespace projects) and organization_repos (list of {org, org_url, repos} buckets per GitLab namespace, matching the GitHub list_remote_repos shape so UIs can flatMap group repos the same way for both providers). Each repo entry includes name, full_name, url, ssh_url, private, description, default_branch, created_at, and updated_at. Optional warnings (array of strings) when group listing partially fails—for example membership scope returns HTTP 403 because the PAT lacks read_api, or a listing reaches the 10,000-project cap. Genesis caps each owned and membership listing independently at 10,000 projects; truncated: true and a warning report when a listing reaches the cap.
• Errors: Detailed error messages with resolution guidance (e.g., authentication failures, merge conflicts, API limits).

​

Best Practices

​

Example: Complete Data Project Workflow

# 1. Store GitLab credentials (one-time setup)
git_action(
    action="store_gitlab_credentials",
    gitlab_token="glpat-abc123xyz..."
)

# 2. Clone existing project repository
git_action(
    action="clone_repo",
    repo_id="customer_analytics",
    url="https://gitlab.com/company/customer-analytics.git"
)

# 3. Create feature branch for new analysis
git_action(
    action="create_branch",
    repo_id="customer_analytics",
    branch_name="feature/churn-prediction"
)

git_action(
    action="switch_branch",
    repo_id="customer_analytics",
    branch_name="feature/churn-prediction"
)

# 4. Create analysis notebook
file(
    action="write",
    repo_id="customer_analytics",
    file_path="notebooks/churn_prediction.py",
    content="""
# Customer Churn Prediction Model
import pandas as pd
from sklearn.ensemble import RandomForestClassifier

# Load and prepare data
customers = pd.read_csv('data/customers.csv')

# Model training logic...
"""
)

# 5. Create CI/CD configuration
file(
    action="write",
    repo_id="customer_analytics",
    file_path=".gitlab-ci.yml",
    content="""
stages:
  - test
  - deploy

test_notebook:
  stage: test
  script:
    - python -m pytest tests/
    - python notebooks/churn_prediction.py --validate

deploy_model:
  stage: deploy
  script:
    - python deploy_model.py
  only:
    - main
"""
)

# 6. Create documentation
file(
    action="write",
    repo_id="customer_analytics",
    file_path="docs/churn_model_methodology.md",
    content="# Churn Prediction Methodology\n\n## Overview\n\nThis model predicts customer churn using..."
)

# 7. Commit changes
git_action(
    action="commit",
    repo_id="customer_analytics",
    commit_message="feat: Add churn prediction model with CI/CD pipeline"
)

# 8. Push to GitLab
git_action(
    action="push",
    repo_id="customer_analytics",
    branch_name="feature/churn-prediction"
)

# 9. Create merge request for review
git_action(
    action="create_pull_request",
    repo_id="customer_analytics",
    branch_name="feature/churn-prediction",
    title="Add Customer Churn Prediction Model",
    body="""## Changes
- Churn prediction model using Random Forest
- Automated testing in CI/CD pipeline
- Comprehensive documentation

## Testing
- Unit tests passing
- Model accuracy: 87%
- Performance benchmarks included

Closes #42"""
)

# 10. Check merge request status
git_action(
    action="get_pull_request_info",
    repo_id="customer_analytics",
    pull_request_number=15
)

# 11. Check repository status
git_action(
    action="get_status",
    repo_id="customer_analytics"
)

​

Advanced Features

​

Bulk File Operations

file(
    action="copy",
    repo_id="customer_analytics",
    source_patterns=["*.sql", "*.py", "*.yml"],
    target_path="archive/v1/",
    preserve_structure=True
)

file(
    action="copy",
    repo_id="customer_analytics",
    source_patterns=["reports/monthly/summary.md"],
    target_path="archive/",
    preserve_structure=True
)
# Writes archive/summary.md

file(
    action="read_multiple",
    repo_id="customer_analytics",
    files=[
        "ddl/customers.sql",
        {"path": "ddl/orders.sql"},
        "models/product_dim.sql",
    ]
)

​

Search Across Files

file(
    action="search",
    repo_id="customer_analytics",
    pattern="SELECT.*FROM customers",
    regex=True
)

​

Fetch Deep History

git_action(
    action="fetch_more_history",
    repo_id="customer_analytics",
    depth=100
)

git_action(
    action="get_history",
    repo_id="customer_analytics",
    max_count=50
)

​

List Remote Repositories

git_action(
    action="list_remote_repos",
    include_orgs=True
)

​

GitLab-Specific Features

​

Merge Request Workflow

• Approval Rules: MRs can require specific approvers before merging
• Pipeline Integration: CI/CD pipelines run automatically on MR creation
• Discussion Threads: Threaded comments that can be resolved
• Draft MRs: Prefix title with “Draft:” or “WIP:” to prevent premature merging
• Squash Commits: Option to squash all commits when merging

# Create draft merge request
git_action(
    action="create_pull_request",
    repo_id="customer_analytics",
    branch_name="feature/experimental",
    title="Draft: Experimental feature testing",
    body="This is a work in progress - do not merge yet"
)

​

Self-Hosted GitLab Support

# Store credentials for custom GitLab instance
git_action(
    action="store_gitlab_credentials",
    gitlab_token="glpat-abc123...",
    # Platform will auto-detect from URL during clone
)

# Clone from self-hosted instance
git_action(
    action="clone_repo",
    repo_id="internal_project",
    url="https://gitlab.company.com/data/internal-project.git"
)

​

Troubleshooting

​

Comparison: GitLab vs GitHub

​

Security Best Practices

​

Enterprise Features

• Code Owners: Automatic reviewer assignment based on file paths
• Approval Rules: Require specific approvers or groups
• Merge Request Dependencies: Block merging until dependent MRs complete
• Security Scanning: SAST, DAST, dependency scanning in CI/CD
• Compliance Frameworks: Enforce policies and audit requirements
• Advanced Analytics: Code review analytics, productivity metrics