Git clone step is very slow

Question & Answer Builds
1

Bitrise Build Issue Report template

Description of the issue

Git clone step takes more than 3 minutes, in my mahcine it takes only 1min, and 8s in circleci for the same repo.

Environment:

Where did the issue happen?

Xcode 9.2.x

Which build Step causes the issue and which version of the step?
git-clone
version: 4.0.5

Reproducibility

  • Does a “Rebuild” help? (You can trigger a rebuild from the Build’s page, by clicking the “Rebuild” button in the top right corner of a finished build) : NO
  • Does a rebuild without caches help? (You can remove the Cache:Pull and Cache:Push steps temporarily to not to use the cache, or you can delete all the caches on the Settings tab of the app. : NO
  • Does the issue happen sporadically, or every time? : Every time
  • Does upgrading the build Step to the latest version help? : NO
  • When did the issue start? :

Local reproduction

No, for my local build it takes 1min

Build log

2

Hi @aminelaadhari,

This is not an issue, just different behaviour.

Why it takes less on Circle CI? Because they cache your source code by default.

By default we don’t store your source code, the source code is only accessed on the build machine, and only in the way you specify it.

If you want us to store your source code you can configure the Build Cache to do so. You can find a related guide at: How to Cache the Source Code directory

Keep in mind that this means:

  1. Everything in the source dir will be stored (in the cache), including artifacts / files generated into the source dir.
  2. You will store a copy of the source code on our servers.

Another alternative might be to specify a Clone Depth for the Git Clone step, so that it won’t fetch the whole history every time.

If you have any questions just let us know! :slight_smile:

3

Hi @viktorbenei, thanks for your response.

Actually, cache pull takes more than 8 minutes to be executed. Is there any guide on how to improve the performance of the cache?

4

Always depends on the setup - I’d suggest you to check the Cache:Push step’s log, it prints what things it caches and how long things take. To optimize the cache you should try to minimize the number of files cached in general.

If you’d need more help feel free to copy paste a related build’s bitrise.io URL (where you have the cache setup) and we’ll try to look into it ASAP :slight_smile:

5

@viktorbenei can you help checking this build? https://www.bitrise.io/build/4bbfb9186e5c5582
Thanks

6

In the config you linked you have the Cache:Pull step after Git Clone. That won’t help with speeding up Git Clone at all, as right now it does a full clean git clone first and only after that it pulls the cache :wink:

If all you’d like to cache is the source code then I’d suggest you to have this step sequence:

  1. Cache:Pull
  2. Git Clone
  3. Cache:Push
  4. Any other step
7

I’ve tried getting the git repo from the cache before the git clone but that does not work because it seems the his results in this error https://github.com/bitrise-steplib/steps-git-clone/issues/86

I think the .git folder is not included in the cache by default? Would it be possible to set this up? We’re cloning a repo from Europe that takes 10 minutes (vs 30 sec when pulled from github in the US)…

8

Hi @teameh! The repo is not cached by default, though if you manually set up caching for that directory in the step options, I don’t believe we have any special filter in place to ignore the .git directory. If you could provide the build URL (app.bitrise.io/build/<build-slug>) where you’re experiencing the issue, I’d be happy to look into it.