How to remove contents from Git when hardware limit was reached ?

[Thibault Derrien]

Dear support,

My repository is currently blocked as I reached the 2 GB limit. However, I cannot delete some files anymore, although I have followed the guide given by Bitbucket.

Unfortunately, I have to present my results in 2 days in a very important congress, and I need an access to my Bitbucket repository as early as possible.

Many thanks for your help and best regards,
Thibault Derrien

[lsimont]

Hi Abhin,

I ran into the same problem as Abhin and can't push the changes because my repo is over 2 gb. Could you also run a GC on mine?

Thank you!

[Thibault Derrien]

Hi! Thanks for your action.

I think I would prefer to abandon Git LFS, and remove binary files from my repository.

But then it means that files are still taking too much space. Have to clean more. And then clean history, indeed.

I will tell when it's ready.

[abhin-atl]

Hi Thibault,

I ran the aggressive GC, but it only decreased the repo size marginally. Perhaps you need to move more files into LFS and clean up your history?

[Thibault Derrien]

Hello,

Finally, the limit of 2GB is not sufficient for me, then I have removed my large files from the repository and will manage with TAR and SCP. Can I ask you to run an aggressive git gc, again?

Many thanks,
Thibault

[Thibault Derrien]

I use this repository from several places, including clusters. It looks that I did not succeed to synchronize them. Can I ask you to again make an aggressive gc ?

[abhin-atl]

Removing history in Git involves creating new commits, moving existing refs to point at them and pushing everything. That still means that the old commits are still in the history. They are removed when we run GC on the server. So further cleanups still require GC. Now usually, Git GC runs automatically on the servers, but it was just faster for me to trigger it manually for now.

Using LFS only means that any future large files will not be a part of the core repo history. The existing commits with references to existing blobs will still be a part of your repo. Using LFS will help the repository size from growing in the future.

I recommend watching https://www.youtube.com/watch?v=iJ3hXuEVKAc to understand LFS and how it works. It's also kinda fascinating.

[Thibault Derrien]

Great. I will continue the washing up.
If I add files to Git LFS, can it decrease the Git repository size also? Or other GC will be needed?

[abhin-atl]

I ran the aggressive GC. The size is now lower than 2GB.

[Thibault Derrien]

Hello Abhin,

Many thanks for your help!
I have pushed my changes, and enabled Git LFS to avoid bad situation again. However, it looks that the size of repository has super increased (~ 3 GB now). Can you run an aggressive gc ?

Thanks for your help,
Thibault

[abhin-atl]

Hi Thibault,

When a repo exceeds 2GB, we only accept pushes that are clear deletions of commits. So if any new commit is added, we deny the push. It is for that exact reason that the guide to dealing with reducing the repository size starts with the section Remove the repository limitation, which recommends that you remove the last few commits from BB before running something like BFG.

I just ran an aggressive GC on your repo, which has brought down the size of your repo to 1.8 GB. Please push your post-BFG repo now (the push should work now that the repo size is lowered). Then, when the next GC is run on your repo, the size should reduce quite a bit. Also, rather than waiting for the next GC to run automatically, just ping me (reply to this comment) after you've pushed and I'll run a manual GC again.

[Thibault Derrien]

1/ I have tried to remove files which I don't use and which take space. Then tried to commit and push the changes, but I get always the same answer from repository:

Total 12103 (delta 5275), reused 11562 (delta 4993)
remote: repository is in read only mode (over 2 GB size limit).
To git@bitbucket.org:tjyderrien/octopus.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@bitbucket.org:tjyderrien/octopus.git'

2/ Then I followed the step indicated into the guide for large repositories, i.e.:
I installed and used BFG repo cleaner. I did remove the files bigger than 100 MB (but nothing was found) and I also removed once again the annoying files (all the "aims.out" files which are not useful and take ~ 400 Mo).

I tried to push the changes, it tooks hours, and then was refused by the server.

I don't know what to do. Should I create a new repository to work?

All the best,
Thibault

[tlroche]

I cannot delete some files anymore

What exactly have you tried to do? This is important information for a good bug/problem report.

Particularly, have you tried pushing local changes? e.g., `git rm whatever`, then `git commit -m [why you did it]`, then `git push [remote_branch_name] [local_branch_name]`?