How to programmatically invalidate Bitbucket Pipeline's cache?

Question

I have a node_modules cache in my Bibucket Pipeline and I added new module (eg yarn add react-modal) - how to make Bitbucket pipelines detect new yarn.lock and invalidate its cache?

Taam

I don't think it is possible. I've been trying to work out a number of ways to tackle this, but I came to realise that it's a waste of development time as the package-lock.json does not change that often. I guess it costs less time to invalidate the cache manually if needed rather than 'solve' this issue using all kind of smart tricks, and dependencies will get more stable over time anyway. One thing that might be useful is to add npm install --dryrun to your scripts and break the pipeline if its output does not contain up to date in – Bomke

Here is Atlassian's ticket bitbucket.org/site/master/issues/16314/… , please vote for it, let us have it done and forget! – Taam

@Taam Here's an example of using the internal API to invalidate the cache. Since it's an internal API I would expect it to break and/or behave erratically: github.com/fknussel/atlaskit/blob/master/build/ci-scripts/… – Oxyhydrogen

Thanks @RyanFisher, I will try it as soon I will come back from my sick leave – Taam

@RyanFisher I never had the chance to look at the URL you posted. Today when I finally picked up here, the URL does not work. Could someone provide the content of "clear.pipelines.node.cache.js" file? – Taam

Found it bitbucket.org/atlassian/atlaskit-mk-2/src/master/build/… – Taam

Answer 1

Yeah, as Marecky have already mentioned, there is a ticket for that. Also, there is another one here https://jira.atlassian.com/browse/BCLOUD-17605, which should exactly address the issue. In short, there is an API to invalidate the cache, but it's currently reserved for internal use only.

Elery

Answer 2

Here is the official way to clear the caches

https://bitbucket.org/atlassian/bitbucket-clear-cache/src/master/

Taam

And here the official blog post: bitbucket.org/blog/… – Donegal

Answer 3

Since early 2023, they way to go for the use-case of invalidating a cache on some lockfile changes would be to use the new cache keys feature.

definitions:
  caches:
    node:
      key:
        - yarn.lock
      path: ...

See

• Blog announcement on February 2023
• Documentation

---

Note that the slightest change in the key files will discard the previous cache. This might be sensible in some scenarios but the previous cache might still feature a 99% hit rate, e.g. when adding or updating a single dependency. So, use this if it actually speeds up your use-case and keep in mind that caches are weekly cleared anyway.

Dormie