PubSub: Allow user specified timeout for this executor.submit() (nice to have)

[jeffkpayne]

https://github.com/GoogleCloudPlatform/google-cloud-python/blob/b2c7c11e4e24cf571cd5601803a883542136552c/pubsub/google/cloud/pubsub_v1/subscriber/policy/thread.py#L338

Since the future will block indefinitely, it would be nice to be able to specify a timeout. The current work-around we use is to ensure that the callback function doesn't block forever, but that requires introducing yet another thread construct into the situation.

[jeffkpayne]

@chemelnucfin One thing I noticed was that doesn't seem to be an easy way to pass in any parameters to the thread.Policy used by subscriber.client.Client(). Any customization currently requires the caller to subclass the Policy class. What are your thoughts on how to pass in parameters that then get forwarded to the thread.Policy object instantiated in _get_executor()?

[chemelnucfin]

Hello, sorry for haven't been responding. I believe that my coworker is in the midst of overhauling the pubsub internals, which is why I have not tried to dig deep into pubsub problems. I will keep this in mind and try to find out what is going on with the library. Thanks.

[theacodes]

Pub/Sub internals should be pretty solidified.

@jeffkpayne I'm not sure why this is needed. The future never gets awaited by our code (nor is it exposed to the user in a way that they could await it). We use add_done_callback just to log that the task was completed. Can you tell me more about your use case here?

[jeffkpayne]

Sorry, I worded the issue poorly. Regardless of whether or not future.result() is called, the _callback function that processes the message could block a long time and eat up the threads in the ThreadPoolExecutor. Again, this is a nice-to-have. If this seems unnecessary to you, then please close it.

[theacodes]

Hrm, I may be misunderstanding but it seems undesirable for our executor to kill callbacks. If you're worried your callbacks will hang forever, I feel like that's the responsibility of the author of the callback.

I'm happy to be talked into something different, though.

[jeffkpayne]

OMG, I'm losing my mind. There's no timeout parameter in submit... Closing. Sorry for wasting your time! I need a vacation.

[theacodes]

Hahaha that's quite alright. :)

Hopefully you get to take a vacation soon.