2

i want to have a big concurrencies with the cloud function, so I'm using the cloud function v2. But, in one of the steps, I want only 1 process can do it, while other processes need to wait for it.

Let's say that I have below logic for my function

if (notGenerated()) {
    generateSomething()
}
step2()
step3()
...

If its 'not generated yet', and then there are 2 or more request coming at the same time, how can I block it so that only 1 process/request can do the generation?

In the conventional way, I would use redis as memory cache, to make sure that only 1 process can go through. The other processes could wait, or do any retries.

But if I need to implement the same style to my cloud function, I need to use the memorystore for redis and serverless vps connector, which is not cheap. Especially that in the memorystore, I would be charged for 24 hours, while I might only use it for 3-4 hours only.

How do you check / use the memory cache in cloud function v2? Could I use something like node-cache in cloud function v2? Does the concurrency in cloud function v2 share the same memory? If the cloud functions scale my functions, does it still using the same memory?

Improve this question
4
  • Can you explain more what you are trying to do where you stuck as per guild lines If you are asking about sharing the same memory then your functions can use shared memory between invocations but that memory could also get saturated if you don't clean off the global memory for all functions you can get more info in this video which also recommends when and where to use catching. Commented Jul 18, 2023 at 6:48
  • @RohitKharche sorry if maybe my questions are confusing. To simplify my question, How can I only do the generateSomething() only once, even if my functions are called multiple times in the same time? Commented Jul 18, 2023 at 11:26
  • you could store some value in firestore db and get that value and depending upon the value execute the generateSomething() method value like count and it should be 0 then execute that function. So with this you can break out from concurrency as your function further execution will be judged by that value. Commented Jul 19, 2023 at 5:33
  • @RohitKharche I dont think that this solution works. since that this happens at the same time, then both of the request will get 0 as the count result. then both will generateSomething() at the same time. Commented Jul 20, 2023 at 13:21

1 Answer 1

Reset to default
2
+50

AFAIK your cloud functions don't need to share and probably will not share the same memory or, in general, the same underlying physical hardware between them.

More important, in my opinion, it is advisable to never think in your functions in such way: they are an isolated computed unit that should perform their job taking into account their own physical resources.

If sharing resources is not an option you could try implementing some type of distributed locking/mutex mechanism.

I agree with you that for this purpose Redis is a very suitable use case, although to avoid the costs associated with memorystore you could trying using another alternative approaches.

I am not aware that it will fit your requirements but, extrapolating the idea of filesystem based file locking, you could try using for example a GCS bucket and create blobs on demand. Unfortunately, there is no absolutely guarantee that the operations of checking the existence of the "lock" blob and creating them will be performed by the same function, a race condition could happen, but it may be a suitable solution - although sorry, because even on my own eyes, it doesn't seem an ideal one.

Although I understand your requirements, please, consider modifying your functions to avoid this type of interdependencies, I think that in a certain way it is against their concept and the serverless approach: try conceiving them as idempotent services instead.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I know that the CF (Cloud function) should be idempotent. But in my case, there really is a business logic needs, for 1 of the steps to only run 1 at a time. If the GCS bucket could still generate the race condition, I think it is not doable in my end. I will wait for a while, if there really is no other answer, I think I will accept your answer and could only go with memorystore for redis.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.