pyicat_plus.client.main.IcatClient#

class pyicat_plus.client.main.IcatClient(metadata_urls=None, elogbook_url=None, elogbook_token=None, metadata_queue=None, metadata_queue_monitor_port=None, elogbook_timeout=None, feedback_timeout=None, queue_timeout=None, beamline=None, proposal=None, elogbook_metadata=None, archive_urls=None, archive_queue=None, archive_queue_monitor_port=None, update_metadata_urls=None, update_metadata_queue=None, update_metadata_queue_monitor_port=None, add_files_urls=None, add_files_queue=None, add_files_queue_monitor_port=None, reschedule_investigation_urls=None, reschedule_investigation_queue=None, reschedule_investigation_queue_monitor_port=None, icatplus_restricted_url=None, icatplus_password=None, catalogue_queues=None, catalogue_url=None, tracking_url=None)[source]#

Bases: IcatClientInterface

Client object that provides access to these services:

  • ActiveMQ message broker for creating datasets in ICAT

  • ActiveMQ message broker for updating dataset metadata in ICAT

  • ActiveMQ message broker for updating dataset file count in ICAT

  • ActiveMQ message broker for updating dataset archiving status in ICAT

  • RESTful interface for sending electronic logbook messages/images and get information about investigations

The RESTful interface is referred to as ICAT+ and the ActiveMQ message brokers are consumed by the “ingesters”.

Parameters:

  • metadata_urls (Optional[List[str]])

  • elogbook_url (Optional[str])

  • elogbook_token (Optional[str])

  • metadata_queue (Optional[str])

  • metadata_queue_monitor_port (Optional[int])

  • elogbook_timeout (Optional[float])

  • feedback_timeout (Optional[float])

  • queue_timeout (Optional[float])

  • beamline (Optional[str])

  • proposal (Optional[str])

  • elogbook_metadata (Optional[Mapping])

  • archive_urls (Optional[List[str]])

  • archive_queue (Optional[str])

  • archive_queue_monitor_port (Optional[int])

  • update_metadata_urls (Optional[List[str]])

  • update_metadata_queue (Optional[str])

  • update_metadata_queue_monitor_port (Optional[int])

  • add_files_urls (Optional[List[str]])

  • add_files_queue (Optional[str])

  • add_files_queue_monitor_port (Optional[int])

  • reschedule_investigation_urls (Optional[List[str]])

  • reschedule_investigation_queue (Optional[str])

  • reschedule_investigation_queue_monitor_port (Optional[int])

  • icatplus_restricted_url (Optional[str])

  • icatplus_password (Optional[str])

  • catalogue_queues (Optional[List[str]])

  • catalogue_url (Optional[str])

  • tracking_url (Optional[str])

add_files(dataset_id=None)[source]#

Add missing files to a dataset already ingested.

Parameters:

dataset_id (Optional[int]) – The ICAT dataset identifier.

property add_files_client#
property archive_client#
property catalogue_client#
property current_beamline#
property current_dataset#
property current_dataset_metadata#
property current_path#
property current_proposal#
disconnect()[source]#
do_log_in(password)[source]#

Login to access the restricted part of the API.

Parameters:

password (str)

Return type:

dict

Returns:

authentication info

property elogbook_client#
property expire_datasets_on_close: bool#

A flag indicating whether the dataset expires when it is closed or if it is synchronized with the metadata catalog.

get_investigations_by(filter=None, instrument_name=None, start_date=None, end_date=None)[source]#

Returns a list of investigations matching the provided criteria.

Parameters:

  • filter (Optional[str])

  • instrument_name (Optional[str])

  • start_date (Optional[datetime])

  • end_date (Optional[datetime])

Return type:

List[dict]

Returns:

list of investigations

get_parcels_by(investigation_id)[source]#

Returns the list of parcels associated to an investigation.

Parameters:

investigation_id (str)

Return type:

List[dict]

Returns:

list of parcel information

property investigation_client#
investigation_info(beamline, proposal, date=None, allow_open_ended=True, timeout=None)[source]#

Return the information of the experiment session corresponding to a beamline, proposal and date.

Parameters:

  • beamline (str) – The beamline name.

  • proposal (str) – The proposal name.

  • date (Union[datetime, date, None]) – The date of the proposal, current date by default.

  • allow_open_ended (bool) – If True, enable to select an unofficial experiment session.

  • timeout (Optional[float]) – Set a timeout for the ICAT request.

Return type:

Optional[dict]

Returns:

If found, return the proposal, beamline, e-logbbok url and data portal url of the experiment session.

investigation_info_string(beamline, proposal, date=None, allow_open_ended=True, timeout=None)[source]#

Return the experiment session information as a string.

Parameters:

  • beamline (str) – The beamline name.

  • proposal (str) – The proposal name.

  • date (Union[datetime, date, None]) – The date of the proposal, current date by default.

  • allow_open_ended (bool) – If True, enable to select an unofficial experiment session.

  • timeout (Optional[float]) – Set a timeout for the ICAT request.

Return type:

str

Returns:

If found, return the experiment session information from the metadata catalog as a string.

investigation_summary(beamline, proposal, date=None, allow_open_ended=True, timeout=None)[source]#

Return the experiment session information as a Tuple.

Parameters:

  • beamline (str) – The beamline name.

  • proposal (str) – The proposal name.

  • date (Union[datetime, date, None]) – The date of the proposal, current date by default.

  • allow_open_ended (bool) – If True, enable to select an unofficial experiment session.

  • timeout (Optional[float]) – Set a timeout for the ICAT request.

Return type:

List[Tuple]

Returns:

If found, return the experiment session information from the metadata catalog as a Tuple.

property metadata_client#
property reason_for_missing_information: str#

A string explaining why some information is missing in the metadata catalog.

registered_dataset_ids(beamline, proposal, date=None, allow_open_ended=True, timeout=None)[source]#

Return the dataset list of an experiment session.

Parameters:

  • beamline (str) – The beamline name of the proposal.

  • proposal (str) – The proposal name.

  • date (Union[datetime, date, None]) – The date of the proposal, current date by default.

  • allow_open_ended (bool) – If True, enable to select an unofficial experiment session.

  • timeout (Optional[float]) – Set a timeout for the ICAT request.

Return type:

Optional[List[DatasetId]]

Returns:

The list of datasets (name and path).

registered_datasets(beamline, proposal, date=None, allow_open_ended=True, timeout=None)[source]#

Return the dataset information list of an experiment session.

Parameters:

  • beamline (str) – The beamline name of the proposal.

  • proposal (str) – The proposal name.

  • date (Union[datetime, date, None]) – The date of the proposal, current date by default.

  • allow_open_ended (bool) – If True, enable to select an unofficial experiment session.

  • timeout (Optional[float]) – Set a timeout for the ICAT request.

Return type:

Optional[List[Dataset]]

Returns:

The list of datasets (name, path, ICAT identifier, and DatasetMetadata).

reschedule_investigation(investigation_id)[source]#

Reschedule an investigation defined by its id.

Parameters:

investigation_id (str)

send_binary_data(data, mimetype=None, beamline=None, proposal=None, beamline_only=None, **payload)[source]#

Send an image in base64 format to the proposal or beamline e-logbook.

Parameters:

  • data (bytes) – The binary message content.

  • mimetype (Optional[str]) – {‘text/plain’, ‘text/html’}, optional.

  • beamline (Optional[str]) – The beamline name of the proposal or beamline e-logbook.

  • proposal (Optional[str]) – The proposal name of the e-logbook. Ignored if beamline_only is True.

  • beamline_only (Optional[bool]) – if True, the message will be stored in the beamline e-logbook.

  • payload – Additional payload for the e-logbook message. It can contain tags (list of strings or list of dictionaries), the machine, the software.

send_binary_file(filename, beamline=None, proposal=None, beamline_only=None, **payload)[source]#

Send the content of a file as a binary image to the proposal or beamline e-logbook.

Parameters:

  • filename (str) – The filename of the image to be sent.

  • beamline (Optional[str]) – The beamline name of the proposal or beamline e-logbook.

  • proposal (Optional[str]) – The proposal name of the e-logbook. Ignored if beamline_only is True.

  • beamline_only (Optional[bool]) – if True, the message will be stored in the beamline e-logbook.

  • payload – Additional payload for the e-logbook message. It can contain tags (list of strings or list of dictionaries), the machine, the software.

send_message(msg, msg_type=None, beamline=None, proposal=None, dataset=None, beamline_only=None, editable=None, formatted=None, mimetype=None, **payload)[source]#

Send a message to the proposal or beamline e-logbook.

Parameters:

  • msg (str) – The message content.

  • msg_type (Optional[str]) – {‘comment’, ‘debug’, ‘info’, ‘error’, ‘commandLine’}, optional.

  • beamline (Optional[str]) – The beamline name of the proposal or beamline e-logbook.

  • proposal (Optional[str]) – The proposal name of the e-logbook. Ignored if beamline_only is True.

  • dataset (Optional[str]) – The specific dataset name to link the message to.

  • beamline_only (Optional[bool]) – if True, the message will be stored in the beamline e-logbook.

  • editable (Optional[bool]) – Used with the formatted field, to determine the category of message. Annotation characterizes editable and unformatted messages, while Notification charaterizes non-editable and formatted messages.

  • formatted (Optional[bool]) – Used with the editable field, to determine the category of message. Annotation characterizes editable and unformatted messages, while Notification charaterizes non-editable and formatted messages.

  • mimetype (Optional[str]) – {‘text/plain’, ‘text/html’}, optional.

  • payload – Additional payload for the message. It can contain tags (list of strings or list of dictionaries), the machine, the software.

send_text_file(filename, beamline=None, proposal=None, dataset=None, beamline_only=None, **payload)[source]#

Send the content of a text file as a message to the proposal or beamline e-logbook.

Parameters:

  • filename (str) – The filename containing the message to be sent.

  • beamline (Optional[str]) – The beamline name of the proposal or beamline e-logbook.

  • proposal (Optional[str]) – The proposal name of the e-logbook. Ignored if beamline_only is True.

  • beamline_only (Optional[bool]) – if True, the message will be stored in the beamline e-logbook.

  • payload – Additional payload for the e-logbook message. It can contain tags (list of strings or list of dictionaries), the machine, the software.

  • dataset (Optional[str])

start_investigation(beamline=None, proposal=None, start_datetime=None, end_datetime=None)[source]#

Send a message to ActiveMQ to either synchronize the experiment session from the User Portal in ICAT or to create a test experiment session in ICAT.

Parameters:

  • beamline (Optional[str]) – The beamline name of the proposal.

  • proposal (Optional[str]) – The proposal name.

  • start_datetime – The start date of the experiment session, timezone local time. Current date time by default.

  • end_datetime – The end date of the experiment session, timezone local time.

store_dataset(beamline=None, proposal=None, dataset=None, path=None, metadata=None, store_filename=None)[source]#

Request icat to store raw dataset.

Parameters:

  • str (proposal) – beamline name like id01, id15a, bm18…

  • str – proposal name like in1169, blc14795, ihma429…

  • dataset (Optional[str]) – dataset name.

  • path (Optional[str]) – path to the raw dataset to store. Must be a folder.

  • metadata (Optional[dict]) – metadata to associate to the dataset. Must contains keys defined by the appropriate application definition from https://gitlab.esrf.fr/icat/hdf5-master-config/-/blob/88a975039694d5dba60e240b7bf46c22d34065a0/hdf5_cfg.xml.

  • store_filename (Optional[str]) – xml file with metadata to be stored.

  • beamline (Optional[str])

  • proposal (Optional[str])

  • dataset

  • path

  • metadata

  • store_filename

store_dataset_from_file(store_filename=None)[source]#

Send a message to ActiveMQ to store a dataset and the associated metadata from a xml file stored on the disk.

Parameters:

store_filename (Optional[str]) – The XML filename containing all dataset metadata.

store_processed_data(beamline=None, proposal=None, dataset=None, path=None, metadata=None, raw=(), store_filename=None)[source]#

The ‘raw’ argument is shorthand for metadata = {‘input_datasets’: …}.

Parameters:

  • beamline (Optional[str])

  • proposal (Optional[str])

  • dataset (Optional[str])

  • path (Optional[str])

  • metadata (Optional[dict])

  • raw (Sequence)

  • store_filename (Optional[str])

property tracking_client#
update_archive_restore_status(dataset_id=None, type=None, level=StatusLevel.INFO, message=None)[source]#

Update the archiving or restore status of a dataset.

Parameters:

  • dataset_id (Optional[int]) – The ICAT dataset identifier.

  • type (Optional[StatusType]) – The type of the status, possible options are {‘archiving’, ‘restoration’}.

  • level (StatusLevel) – The level of the status message; possible options are {‘info’, ‘warning’, ‘error’}.

  • message (Optional[str]) – The optional status’ message.

update_metadata(proposal=None, beamline=None, dataset_paths=None, metadata_name=None, metadata_value=None)[source]#

Update or create datasets metadata.

Parameters:

  • proposal (Optional[str]) – The proposal name.

  • beamline (Optional[str]) – The beamline name of the proposal.

  • dataset_paths (Optional[str]) – Comma-separated list of the dataset locations.

  • metadata_name (Optional[str]) – The name of the metadata to update.

  • metadata_value (Optional[str]) – The new value of the metadata.

property update_metadata_client#