Refactoring of backend server and reranker utils.

[iv1111]

• lower log level for document excerpts
• lower log level for score threshold
• moved logging start to earlier time in backend server
• parametrised backend server logging level
• removed unused main function
• removed default function values

[mkumatag]

make sure we bump the version in the makefile and template to not to override the existing image.

[iv1111]

make sure we bump the version in the makefile and template to not to override the existing image.

Bumped up the version.

[dharaneeshvrd]

can we please add the changes to enable debug log for backend server?
Like its enabled for ingestion

project-ai-services/ai-services/assets/applications/rag/values.yaml

Line 15 in 221f6fc

log_level: "INFO"

here

[iv1111]

can we please add the changes to enable debug log for backend server? Like its enabled for ingestion

project-ai-services/ai-services/assets/applications/rag/values.yaml

Line 15 in 221f6fc

log_level: "INFO"

here

This is what you meant? I will check if backend reads it.

[iv1111]

@dharaneeshvrd Looks like back-end already reads this env variable. In future we should probably offload this to settings.json, the corresponding class will handle loading and validation.

https://github.com/iv1111/project-ai-services/blob/fcfbaaef5b8bc97789d3f97b26c61cb647fb680a/spyre-rag/src/retrieve/backend_server.py#L198-L205

[dharaneeshvrd]

@iv1111 once we add the config in values.yaml, we have to refer it in pod deployment like this
Please make necessary changes

[dharaneeshvrd]

@iv1111 can we please remove the default values here

project-ai-services/spyre-rag/src/retrieve/reranker_utils.py

Line 30 in 221f6fc

model: str = "/wca4z-pvc-ckpt/HF_cache/models--BAAI--bge-reranker-large/snapshots/55611d7bca2a7133960a6d3b71e083071bbfc312",

?
@mkumatag hope we can accommodate this minor cleanup!

[Niharika0306]

@iv1111 can we please remove the default values here

project-ai-services/spyre-rag/src/retrieve/reranker_utils.py

Line 30 in 221f6fc

model: str = "/wca4z-pvc-ckpt/HF_cache/models--BAAI--bge-reranker-large/snapshots/55611d7bca2a7133960a6d3b71e083071bbfc312",

?
@mkumatag hope we can accommodate this minor cleanup!

+1

[mkumatag]

@iv1111 can we please remove the default values here

project-ai-services/spyre-rag/src/retrieve/reranker_utils.py

Line 30 in 221f6fc

model: str = "/wca4z-pvc-ckpt/HF_cache/models--BAAI--bge-reranker-large/snapshots/55611d7bca2a7133960a6d3b71e083071bbfc312",

?
@mkumatag hope we can accommodate this minor cleanup!

we are always overriding it right? and there is one more line post that also need to be altered. If functionality wise it is not making any impact - please avoid making any change for now.

[dharaneeshvrd]

yes @Niharika0306 please confirm once.

[Niharika0306]

@iv1111 can we please remove the default values here

project-ai-services/spyre-rag/src/retrieve/reranker_utils.py

Line 30 in 221f6fc

model: str = "/wca4z-pvc-ckpt/HF_cache/models--BAAI--bge-reranker-large/snapshots/55611d7bca2a7133960a6d3b71e083071bbfc312",

?
@mkumatag hope we can accommodate this minor cleanup!

we are always overriding it right? and there is one more line post that also need to be altered. If functionality wise it is not making any impact - please avoid making any change for now.

Yes, we are always overriding it. These are some public instance links that was pointed by research team.
but yes, as you said - its not super important now.

[dharaneeshvrd]

@iv1111 We have to move this block https://github.com/IBM/project-ai-services/blob/main/spyre-rag/src/retrieve/backend_server.py#L198toL205 before importing other modules, to propagate the log level to those modules, can you please take care of this as well?

[iv1111]

@iv1111 We have to move this block https://github.com/IBM/project-ai-services/blob/main/spyre-rag/src/retrieve/backend_server.py#L198toL205 before importing other modules, to propagate the log level to those modules, can you please take care of this as well?

@dharaneeshvrd Done, PTAL.

Yes, we are always overriding it. These are some public instance links that was pointed by research team.
but yes, as you said - its not super important now.

@Niharika0306 but it looks like here we call it without passing last three params relying on default values: https://github.com/iv1111/project-ai-services/blob/2a865c06399b6be6d80548c7929a583b59736128/spyre-rag/src/retrieve/reranker_utils.py#L69
@mkumatag Since this PR won't go into GA1, thus I could handle the change here right away.

yes, which is called only when the script is invoked separately. Nonetheless, if this PR is not being merged for GA1, we can go ahead and remove the call and the defaults too.

I removed the default values from reranker utils. As for invocation, I removed the whole main function there, it is not needed.

[iv1111]

@dharaneeshvrd, @Niharika0306 PTAL and each approve if changes are ok.

[dharaneeshvrd]

make sure we bump the version in the makefile and template to not to override the existing image.

please do this.
Otherwise lgtm, will approve post above change.

[dharaneeshvrd]

Need approval from @mkumatag