ADO Integration to ML Studio.

Hi Srinivas Gajula

The comprehensive solution for resolving persistent Git clone issues combines multiple approaches with detailed monitoring capabilities. Here are several approaches to resolve the "premature end of pack file" and "invalid index-pack output" errors when cloning the repository into your ML Studio compute instance:

1.Shallow Clone Approach

# First, create and initialize a new repository
mkdir COI-CAAI-PygoraService
cd COI-CAAI-PygoraService
git init

# Add the remote repository
git remote add origin https://msazure.visualstudio.com/One/_git/COI-CAAI-PygoraService

# Fetch with limited history
git fetch --depth=1 origin main

# Checkout the main branch
git checkout origin/main

# Optional: Get more history if needed
git fetch --depth=100
git fetch --unshallow

This approach uses shallow cloning to reduce the initial data transfer. By fetching only recent commits first, it avoids the premature EOF errors that often occur with large repositories over unstable connections. The --depth=1 flag limits the clone to just the latest commit, making it much more reliable for initial setup.

2. Modified Git Configuration Approach

# Set optimized git configuration
git config --global http.postBuffer 1048576000
git config --global core.compression 0
git config --global pack.windowMemory 50m

# Clone with authentication token
git -c http.extraheader="AUTHORIZATION: bearer $accessToken" \
    clone https://msazure.visualstudio.com/One/_git/COI-CAAI-PygoraService

This solution optimizes Git's configuration settings to handle larger transfers more effectively. The increased postBuffer size helps prevent timeouts, while disabling compression reduces memory usage during transfer. These settings are particularly helpful for repositories with large files or many objects.

For Git Clone Network issues we need to follow the concise steps

#!/bin/bash
RETRY_COUNT=5
DELAY_SECONDS=30
for attempt in $(seq 1 $RETRY_COUNT); do
    echo "$(date '+%Y-%m-%d %H:%M:%S') - Attempt $attempt/$RETRY_COUNT"
    
    TMP_DIR=$(mktemp -d)
    cd "$TMP_DIR"
    git init
    
    git config --global http.postBuffer 1048576000
    git config --global core.compression 0
    git config --global pack.windowMemory 50m
    
    export GIT_TRACE=1
    export GIT_TRACE_PACKET=1
    export GIT_CURL_VERBOSE=1
    
    if git fetch --depth=1 origin main 2>&1 | tee "clone_attempt_$attempt.log"; then
        exit 0
    fi
    
    sleep "$DELAY_SECONDS"
done

We can monitor the progress with below commands

sudo tcpdump -i any -vvv -s 0 -c 100 port 443 &
top -b -d 1 > system_resources.log &

Hope it helps!

Thank you