lce_service.sf_tools_api.retrieve_tools module

class lce_service.sf_tools_api.retrieve_tools.SFToolResult(result: Any, extracted: Dict[str, Any] | None, error: str | None, chunk_count: int, incomplete: bool = False)

Bases: NamedTuple

The result a finished Snippet Framework tool wrote, reassembled and parsed.

Variables:

  • result – The parsed JSON result exactly as the CLI wrote it, or None when there is none.

  • extracted – The fields extract_sf_tool_result pulls out of the result, or None.

  • error – Why no result is available, or None when one is. A tool that finishes without a result, as the CLI does when it finds no jobs to run, reports why in its log.

  • chunk_count – How many chunks the result arrived in.

  • incomplete – Whether the chunks that were found do not make a whole result, because one has a length the tool cannot write or the text does not parse. Until the job has been quiet for SETTLE_SECONDS that can mean a batch of chunks is still on its way rather than that it was lost, so callers treat such a result as not yet available rather than as final.

as_dict() → Dict[str, Any]

Returns the result as it is reported to a caller.

Returns:

A dictionary of the result, the extracted fields and the error.

chunk_count: int

Alias for field number 3

error: str | None

Alias for field number 2

extracted: Dict[str, Any] | None

Alias for field number 1

incomplete: bool

Alias for field number 4

result: Any

Alias for field number 0

lce_service.sf_tools_api.retrieve_tools.assert_sf_tool_job_is_readable(dbc: silo_cursor, proc_id: int, tool: SFTool, user: User) → Dict[str, Any]

Verifies that a process id names a job of this tool that this user queued.

A job is only readable through the endpoint for the tool it was queued as. The output table is shared by every Remote Toolbox tool, so a job of any other tool, including another of the Snippet Framework tools, is refused exactly as a job that does not exist is.

Parameters:

  • dbc – A silo_cursor instance used to read the spool and ownership rows.

  • proc_id – The process id returned when the tool was queued.

  • tool – The tool named in the request path.

  • user – The User instance requesting the output.

Returns:

The job’s spool row, so the caller does not have to read it again.

Raises:

  • SFToolJobNotAccessible – Raised when no such job exists, when it was queued as a different tool, when it has no recorded owner, or when the owner is another user.

  • DatabaseExecutionError – Raised when either check fails to execute.

lce_service.sf_tools_api.retrieve_tools.poll_sf_tool_output(connect: Callable[[], ContextManager[silo_cursor]], proc_id: int, tool: SFTool, user: User, after_id: int = 0, page_size: int | None = None, wait_seconds: float | None = None) → Tuple[OutputPage, JobProgress]

Serves one poll of a tool run: checks access, then reads a page of its log and its progress.

The result chunks are left out of the page and out of the progress. They are read whole by read_sf_tool_result once the run has finished, so they neither spend the page budget, nor get cut by the per record cap, nor appear to a caller as lines of log, and the last output id the progress reports is one a reader of the log can reach. A poll with no wait uses a single connection; a wait opens one per read, as poll_job_output describes.

Parameters:

  • connect – Opens a database connection for one read, such as db_manager.

  • proc_id – The process id returned when the tool was queued.

  • tool – The tool named in the request path, which the job must have been queued as.

  • user – The User instance requesting the output.

  • after_id – Return only rows with an id greater than this.

  • page_size – The maximum number of rows to return.

  • wait_seconds – How long to wait for the collector to finish. Defaults to no wait.

Returns:

The page of log lines and the run’s progress, read after it.

Raises:
lce_service.sf_tools_api.retrieve_tools.read_local_sf_tool_result(run: LocalRun, tool: SFTool) → SFToolResult

Reassembles and parses the JSON result a tool run on this host wrote.

The result is read from the rows the run produced by the same rules read_sf_tool_result applies to the rows a collector wrote, so a run gives the same result either way.

Parameters:

  • run – The LocalRun of the tool.

  • tool – The tool that was run.

Returns:

An SFToolResult, as read_sf_tool_result returns it. A run whose output passed the limit a local run keeps has lost part of its result, so it is withheld.

lce_service.sf_tools_api.retrieve_tools.read_sf_tool_result(dbc: silo_cursor, proc_id: int, tool: SFTool) → SFToolResult

Reassembles and parses the JSON result a finished Snippet Framework tool wrote.

The CLI cuts its JSON into fixed length chunks and writes each behind CLI_OUTPUT_HEADER, so the result is rebuilt from those rows alone, whole and in order, before it is parsed. The platform’s editor trims every chunk before joining them, which deletes whitespace inside string values wherever a chunk boundary falls next to it; only the line terminator is removed here.

The CLI cuts every chunk but the last to exactly CLI_OUTPUT_CHUNK_SIZE characters, so a chunk of any other length means part of the result was lost or mangled on its way to the database; the collector’s toolbox, for one, can lose the start of a line it was half way through reading when the command exited. Such a result is withheld even when what remains still parses, as it will whenever the loss falls inside a long string, because a result that parses is not the same as the result the tool wrote. A chunk lost whole cannot be detected this way, since the CLI does not say how many it wrote.

Parameters:

  • dbc – A silo_cursor instance used to read the output rows.

  • proc_id – The process id returned when the tool was queued.

  • tool – The tool the job was queued as.

Returns:

An SFToolResult. Its error is set, and its result None, when the tool wrote no result, wrote one too large to return, or wrote one that does not parse.

Raises:

DatabaseExecutionError – Raised when the output rows cannot be read.