.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/lemon_example.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_lemon_example.py: ====================== LEMON dataset example ====================== This example illustrates the use of ``sovabids`` on the `LEMON dataset `_ using both the python API and the CLI tool. The main elements of this example are: * A source path with the original dataset. * A bids path that will be the output path of the conversion. * A rules file that configures how the conversion is done. * A mapping file that encodes how the conversion is performed to each individual file of the dataset. .. mermaid:: graph LR S>"Source path"] B>"Bids path"] R>"Rules file"] AR(("Apply Rules")) M>"Mappings file"] CT(("Convert Them")) O[("Converted dataset")] S --> AR B --> AR R --> AR AR --> M M --> CT CT --> O .. GENERATED FROM PYTHON SOURCE LINES 35-42 Using the python API -------------------- First we will illustrate how to run the software within python. Imports ^^^^^^^ First we import some functions we will need: .. GENERATED FROM PYTHON SOURCE LINES 42-50 .. code-block:: Python import os # For path manipulation import shutil # File manipulation from mne_bids import print_dir_tree # To show the input/output directories structures inside this example from sovabids.rules import apply_rules # Apply rules for conversion from sovabids.convert import convert_them # Do the conversion from sovabids.datasets import lemon_prepare # Download the dataset from sovabids.settings import REPO_PATH .. GENERATED FROM PYTHON SOURCE LINES 51-58 Getting and preparing the dataset ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ We have to download and decompress the dataset. We also need to fix a filename inconsistency (without this correction the file won't be able to be opened in mne). Luckily all of that is encapsulated in the lemon_prepare function since these issues are not properly of sovabids. By default the files are saved in the '_data' directory of the sovabids project. .. GENERATED FROM PYTHON SOURCE LINES 58-60 .. code-block:: Python lemon_prepare() .. rst-class:: sphx-glr-script-out .. code-block:: none Downloading sub-032301.tar.gz at /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon WARNING: File already existed. Skipping... Downloading sub-032302.tar.gz at /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon WARNING: File already existed. Skipping... Downloading sub-032303.tar.gz at /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon WARNING: File already existed. Skipping... Downloading name_match.csv at /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon WARNING: File already existed. Skipping... LEMON PREPARE DONE! .. GENERATED FROM PYTHON SOURCE LINES 61-66 Setting up the paths ^^^^^^^^^^^^^^^^^^^^ Now we will set up four paths. Because this example is intended to run relative to the repository directory we use relative path but for real use-cases it is easier to just input the absolute-path. We will print these paths for more clarity. .. GENERATED FROM PYTHON SOURCE LINES 66-77 .. code-block:: Python source_path = os.path.abspath(os.path.join(REPO_PATH,'_data','lemon')) # For the input data we will convert bids_path= os.path.abspath(os.path.join(REPO_PATH,'_data','lemon_bids')) # The output directory that will have the converted data rules_path = os.path.abspath(os.path.join(REPO_PATH,'examples','lemon_example_rules.yml')) # The rules file that setups the rule for conversion mapping_path = os.path.abspath(os.path.join(bids_path,'code','sovabids','mappings.yml')) # The mapping file that will hold the results of applying the rules to each file print('source_path:',source_path.replace(REPO_PATH,'')) print('bids_path:', bids_path.replace(REPO_PATH,'')) print('rules_path:',rules_path.replace(REPO_PATH,'')) print('mapping_path:',mapping_path.replace(REPO_PATH,'')) .. rst-class:: sphx-glr-script-out .. code-block:: none source_path: /_data/lemon bids_path: /_data/lemon_bids rules_path: /examples/lemon_example_rules.yml mapping_path: /_data/lemon_bids/code/sovabids/mappings.yml .. GENERATED FROM PYTHON SOURCE LINES 78-81 Cleaning the output directory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ We will clean the output path as a safety measure from previous conversions. .. GENERATED FROM PYTHON SOURCE LINES 81-87 .. code-block:: Python try: shutil.rmtree(bids_path) except: pass .. GENERATED FROM PYTHON SOURCE LINES 88-91 The input directory ^^^^^^^^^^^^^^^^^^^ For clarity purposes we will print here the directory we are trying to convert to BIDS. .. GENERATED FROM PYTHON SOURCE LINES 91-94 .. code-block:: Python print_dir_tree(source_path) .. rst-class:: sphx-glr-script-out .. code-block:: none |lemon/ |--- name_match.csv |--- sub-032301.tar.gz |--- sub-032302.tar.gz |--- sub-032303.tar.gz |--- sub-010002/ |------ RSEEG/ |--------- sub-010002.eeg |--------- sub-010002.vhdr |--------- sub-010002.vmrk |--- sub-010003/ |------ RSEEG/ |--------- sub-010003.eeg |--------- sub-010003.vhdr |--------- sub-010003.vmrk |--- sub-010004/ |------ RSEEG/ |--------- sub-010004.eeg |--------- sub-010004.vhdr |--------- sub-010004.vmrk .. GENERATED FROM PYTHON SOURCE LINES 95-106 Making the rules ^^^^^^^^^^^^^^^^ The most important and complicated part of this is making the rules file, either by hand or by the "DISCOVER_RULES" module (which is not yet implemented). This part is already done for you, but for clarification here are the rules we are applying. Please read the following output as the yaml has some basic documentation comments. See the Rules File Schema documentation for help regarding making this rules file. .. GENERATED FROM PYTHON SOURCE LINES 106-110 .. code-block:: Python with open(rules_path,encoding="utf-8") as f: rules = f.read() print(rules) .. rst-class:: sphx-glr-script-out .. code-block:: none entities: # Configuring the file name structure of bids task : resting # Setting the task of all files to a fixed string dataset_description: # Configuring the dataset_description.json file Name : Lemon # Name of the dataset, set up as a fixed string Authors: # Here I put the personnel involved in the acquisition of the dataset - Anahit Babayan # See http://fcon_1000.projects.nitrc.org/indi/retro/MPI_LEMON.html - Miray Erbey - Deniz Kumral - Janis D. Reinelt - Andrea M. F. Reiter - Josefin Röbbig - H. Lina Schaare - Marie Uhlig - Alfred Anwander - Pierre-Louis Bazin - Annette Horstmann - Leonie Lampe - Vadim V. Nikulin - Hadas Okon-Singer - Sven Preusser - André Pampel - Christiane S. Rohr - Julia Sacher1 - Angelika Thöne-Otto - Sabrina Trapp - Till Nierhaus - Denise Altmann - Katrin Arelin - Maria Blöchl - Edith Bongartz - Patric Breig - Elena Cesnaite - Sufang Chen - Roberto Cozatl - Saskia Czerwonatis - Gabriele Dambrauskaite - Maria Dreyer - Jessica Enders - Melina Engelhardt - Marie Michele Fischer - Norman Forschack - Johannes Golchert - Laura Golz - C. Alexandrina Guran - Susanna Hedrich - Nicole Hentschel - Daria I. Hoffmann - Julia M. Huntenburg - Rebecca Jost - Anna Kosatschek - Stella Kunzendorf - Hannah Lammers - Mark E. Lauckner - Keyvan Mahjoory - Natacha Mendes - Ramona Menger - Enzo Morino - Karina Näthe - Jennifer Neubauer - Handan Noyan - Sabine Oligschläger - Patricia Panczyszyn-Trzewik - Dorothee Poehlchen - Nadine Putzke - Sabrina Roski - Marie-Catherine Schaller - Anja Schieferbein - Benito Schlaak - Hanna Maria Schmidt - Robert Schmidt - Anne Schrimpf - Sylvia Stasch - Maria Voss - Anett Wiedemann - Daniel S. Margulies - Michael Gaebler - Arno Villringer sidecar: # Configuring the sidecar eeg file PowerLineFrequency : 50 # Noted from the visual inspection of the eeg spectrum EEGReference : FCz # As mentioned in https://www.nature.com/articles/sdata2018308 channels: # Configuring the channels tsv type : # This property allow us to overwrite channel types inferred by MNE VEOG : VEOG # Here the syntax is : F3 : EEG # Here we set the type of F3, it was already correctly inferred by mne but it is included to illustrate retyping of various channels. non-bids: # Additional configuration not belonging specifically to any of the previous objects eeg_extension : .vhdr # Sets which extension to read as an eeg file path_analysis: # Some bids properties can be inferred from the path of the source files pattern : RSEEG/sub-%entities.subject%.vhdr # For example here we extract from the path the "subject" child of the "entities" object .. GENERATED FROM PYTHON SOURCE LINES 111-118 Applying the rules ^^^^^^^^^^^^^^^^^^ We apply the rules to the input dataset by giving the input,ouput,rules, and mapping paths to the apply_rules function. This will produce by default a 'mappings.yml' file at the specified directory of 'bids_path/code/sovabids'. This file holds the result of applying the rules to each of the dataset files. .. GENERATED FROM PYTHON SOURCE LINES 118-120 .. code-block:: Python apply_rules(source_path,bids_path,rules_path,mapping_path) .. rst-class:: sphx-glr-script-out .. code-block:: none Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010002/RSEEG/sub-010002.vhdr... Setting channel info structure... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010003/RSEEG/sub-010003.vhdr... Setting channel info structure... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010004/RSEEG/sub-010004.vhdr... Setting channel info structure... {'General': {'entities': {'task': 'resting'}, 'dataset_description': {'Name': 'Lemon', 'Authors': ['Anahit Babayan', 'Miray Erbey', 'Deniz Kumral', 'Janis D. Reinelt', 'Andrea M. F. Reiter', 'Josefin Röbbig', 'H. Lina Schaare', 'Marie Uhlig', 'Alfred Anwander', 'Pierre-Louis Bazin', 'Annette Horstmann', 'Leonie Lampe', 'Vadim V. Nikulin', 'Hadas Okon-Singer', 'Sven Preusser', 'André Pampel', 'Christiane S. Rohr', 'Julia Sacher1', 'Angelika Thöne-Otto', 'Sabrina Trapp', 'Till Nierhaus', 'Denise Altmann', 'Katrin Arelin', 'Maria Blöchl', 'Edith Bongartz', 'Patric Breig', 'Elena Cesnaite', 'Sufang Chen', 'Roberto Cozatl', 'Saskia Czerwonatis', 'Gabriele Dambrauskaite', 'Maria Dreyer', 'Jessica Enders', 'Melina Engelhardt', 'Marie Michele Fischer', 'Norman Forschack', 'Johannes Golchert', 'Laura Golz', 'C. Alexandrina Guran', 'Susanna Hedrich', 'Nicole Hentschel', 'Daria I. Hoffmann', 'Julia M. Huntenburg', 'Rebecca Jost', 'Anna Kosatschek', 'Stella Kunzendorf', 'Hannah Lammers', 'Mark E. Lauckner', 'Keyvan Mahjoory', 'Natacha Mendes', 'Ramona Menger', 'Enzo Morino', 'Karina Näthe', 'Jennifer Neubauer', 'Handan Noyan', 'Sabine Oligschläger', 'Patricia Panczyszyn-Trzewik', 'Dorothee Poehlchen', 'Nadine Putzke', 'Sabrina Roski', 'Marie-Catherine Schaller', 'Anja Schieferbein', 'Benito Schlaak', 'Hanna Maria Schmidt', 'Robert Schmidt', 'Anne Schrimpf', 'Sylvia Stasch', 'Maria Voss', 'Anett Wiedemann', 'Daniel S. Margulies', 'Michael Gaebler', 'Arno Villringer']}, 'sidecar': {'PowerLineFrequency': 50, 'EEGReference': 'FCz'}, 'channels': {'type': {'VEOG': 'VEOG', 'F3': 'EEG'}}, 'non-bids': {'eeg_extension': '.vhdr', 'path_analysis': {'pattern': 'RSEEG/sub-%entities.subject%.vhdr'}}, 'IO': {'source': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon', 'target': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids'}}, 'Individual': [{'non-bids': {'eeg_extension': '.vhdr', 'path_analysis': {'pattern': 'RSEEG/sub-%entities.subject%.vhdr'}}, 'sidecar': {'PowerLineFrequency': 50, 'EEGReference': 'FCz'}, 'channels': {'type': {'VEOG': 'VEOG', 'F3': 'EEG'}}, 'entities': {'subject': '010002', 'task': 'resting'}, 'IO': {'target': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_eeg.vhdr', 'source': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010002/RSEEG/sub-010002.vhdr'}}, {'non-bids': {'eeg_extension': '.vhdr', 'path_analysis': {'pattern': 'RSEEG/sub-%entities.subject%.vhdr'}}, 'sidecar': {'PowerLineFrequency': 50, 'EEGReference': 'FCz'}, 'channels': {'type': {'VEOG': 'VEOG', 'F3': 'EEG'}}, 'entities': {'subject': '010003', 'task': 'resting'}, 'IO': {'target': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_eeg.vhdr', 'source': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010003/RSEEG/sub-010003.vhdr'}}, {'non-bids': {'eeg_extension': '.vhdr', 'path_analysis': {'pattern': 'RSEEG/sub-%entities.subject%.vhdr'}}, 'sidecar': {'PowerLineFrequency': 50, 'EEGReference': 'FCz'}, 'channels': {'type': {'VEOG': 'VEOG', 'F3': 'EEG'}}, 'entities': {'subject': '010004', 'task': 'resting'}, 'IO': {'target': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_eeg.vhdr', 'source': '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010004/RSEEG/sub-010004.vhdr'}}]} .. GENERATED FROM PYTHON SOURCE LINES 121-124 Doing the conversion ^^^^^^^^^^^^^^^^^^^^ We now do the conversion of the dataset by reading the mapping file ('mappings.yml') with the convert them module. .. GENERATED FROM PYTHON SOURCE LINES 124-126 .. code-block:: Python convert_them(mapping_path) .. rst-class:: sphx-glr-script-out .. code-block:: none Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010002/RSEEG/sub-010002.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/README'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/no USB Connection to actiCAP', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_channels.tsv'... Copying data files to sub-010002_task-resting_eeg.vhdr /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Converting data files to BrainVision format write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Encountered data in "short" format. Converting to float32. write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/sub-010002_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/sub-010002_scans.tsv entry with eeg/sub-010002_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010002/eeg/sub-010002_task-resting_eeg.json'... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010003/RSEEG/sub-010003.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/no USB Connection to actiCAP', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_channels.tsv'... Copying data files to sub-010003_task-resting_eeg.vhdr /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Converting data files to BrainVision format write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Encountered data in "short" format. Converting to float32. write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/sub-010003_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/sub-010003_scans.tsv entry with eeg/sub-010003_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010003/eeg/sub-010003_task-resting_eeg.json'... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010004/RSEEG/sub-010004.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/actiCAP Data On', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_channels.tsv'... Copying data files to sub-010004_task-resting_eeg.vhdr /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Converting data files to BrainVision format write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/sovabids/rules.py:278: RuntimeWarning: Encountered data in "short" format. Converting to float32. write_raw_bids(raw, bids_path=bids_path,format=output_format,allow_preload=True,overwrite=True) Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/sub-010004_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/sub-010004_scans.tsv entry with eeg/sub-010004_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/sub-010004/eeg/sub-010004_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids/dataset_description.json'... .. GENERATED FROM PYTHON SOURCE LINES 127-130 Checking the conversion ^^^^^^^^^^^^^^^^^^^^^^^ For clarity purposes we will check the output directory we got from sovabids. .. GENERATED FROM PYTHON SOURCE LINES 130-135 .. code-block:: Python print_dir_tree(bids_path) print('LEMON CONVERSION FINISHED!') .. rst-class:: sphx-glr-script-out .. code-block:: none |lemon_bids/ |--- README |--- dataset_description.json |--- participants.json |--- participants.tsv |--- code/ |------ sovabids/ |--------- mappings.yml |--------- sovabids.log |--------- sovabids.log.errors |--- sub-010002/ |------ sub-010002_scans.tsv |------ eeg/ |--------- sub-010002_task-resting_channels.tsv |--------- sub-010002_task-resting_eeg.eeg |--------- sub-010002_task-resting_eeg.json |--------- sub-010002_task-resting_eeg.vhdr |--------- sub-010002_task-resting_eeg.vmrk |--------- sub-010002_task-resting_events.json |--------- sub-010002_task-resting_events.tsv |--- sub-010003/ |------ sub-010003_scans.tsv |------ eeg/ |--------- sub-010003_task-resting_channels.tsv |--------- sub-010003_task-resting_eeg.eeg |--------- sub-010003_task-resting_eeg.json |--------- sub-010003_task-resting_eeg.vhdr |--------- sub-010003_task-resting_eeg.vmrk |--------- sub-010003_task-resting_events.json |--------- sub-010003_task-resting_events.tsv |--- sub-010004/ |------ sub-010004_scans.tsv |------ eeg/ |--------- sub-010004_task-resting_channels.tsv |--------- sub-010004_task-resting_eeg.eeg |--------- sub-010004_task-resting_eeg.json |--------- sub-010004_task-resting_eeg.vhdr |--------- sub-010004_task-resting_eeg.vmrk |--------- sub-010004_task-resting_events.json |--------- sub-010004_task-resting_events.tsv LEMON CONVERSION FINISHED! .. GENERATED FROM PYTHON SOURCE LINES 136-160 Using the CLI tool ------------------ sovabids can also be used through the command line. Here we provide an example of how to do so. The overview of what we are doing is now: .. mermaid:: graph LR S>"Source path"] B>"Bids path"] R>"Rules file"] AR(("sovapply")) M>"Mappings file"] CT(("sovaconvert")) O[("Converted dataset")] S --> AR B --> AR R --> AR AR --> M M --> CT CT --> O .. GENERATED FROM PYTHON SOURCE LINES 162-175 Same old blues ^^^^^^^^^^^^^^ Notice that we will run this inside of python so that the example can be run without needing configuration. To run this locally you will need to run lemon_prepare() function from the command line. You can do so by running: .. code-block:: bash python -c "from sovabids.datasets import lemon_prepare; lemon_prepare()" Since we already have run lemon_prepare() inside this example, we will start from this step. We set up the paths again, but now we will change the output to a new path (with "_cli" at the end). We will also clean this path as we did before. .. GENERATED FROM PYTHON SOURCE LINES 175-190 .. code-block:: Python source_path = os.path.abspath(os.path.join(REPO_PATH,'_data','lemon')) # For the input data we will convert bids_path= os.path.abspath(os.path.join(REPO_PATH,'_data','lemon_bids_cli')) # The output directory that will have the converted data rules_path = os.path.abspath(os.path.join(REPO_PATH,'examples','lemon_example_rules.yml')) # The rules file that setups the rule for conversion mapping_path = os.path.abspath(os.path.join(bids_path,'code','sovabids','mappings.yml')) # The mapping file that will hold the results of applying the rules to each file print('source_path:',source_path.replace(REPO_PATH,'')) print('bids_path:', bids_path.replace(REPO_PATH,'')) print('rules_path:',rules_path.replace(REPO_PATH,'')) print('mapping_path:',mapping_path.replace(REPO_PATH,'')) try: shutil.rmtree(bids_path) except: pass .. rst-class:: sphx-glr-script-out .. code-block:: none source_path: /_data/lemon bids_path: /_data/lemon_bids_cli rules_path: /examples/lemon_example_rules.yml mapping_path: /_data/lemon_bids_cli/code/sovabids/mappings.yml .. GENERATED FROM PYTHON SOURCE LINES 191-194 Some necessary code ^^^^^^^^^^^^^^^^^^^ To be able to run commands from this notebook and capture their outputs we need to define the following, nevertheless this is not relevant to actually running this from the command line. .. GENERATED FROM PYTHON SOURCE LINES 194-204 .. code-block:: Python from subprocess import PIPE, run def out(command): result = run(command, stdout=PIPE, stderr=PIPE, universal_newlines=True, shell=True) return result.stdout my_output = out("echo hello world") print(my_output) .. rst-class:: sphx-glr-script-out .. code-block:: none hello world .. GENERATED FROM PYTHON SOURCE LINES 205-211 sovapply ^^^^^^^^ In this example we have already made the rules. So we will apply them using the sovapply tool. Use the following command to print the help of the tool: .. GENERATED FROM PYTHON SOURCE LINES 211-213 .. code-block:: Python command = "sovapply --help" print(command) .. rst-class:: sphx-glr-script-out .. code-block:: none sovapply --help .. GENERATED FROM PYTHON SOURCE LINES 214-216 This will give the following output .. GENERATED FROM PYTHON SOURCE LINES 216-219 .. code-block:: Python my_output= out(command) print(my_output) .. rst-class:: sphx-glr-script-out .. code-block:: none usage: sovapply apply_rules [-h] [-m MAPPING] source_path bids_path rules positional arguments: source_path The path to the input data directory that will be converted to bids bids_path The path to the output bids directory rules The fullpath of the rules file optional arguments: -h, --help show this help message and exit -m MAPPING, --mapping MAPPING The fullpath of the mapping file to be written. If not set it will be located in bids_path/code/sovabids/mappings.yml .. GENERATED FROM PYTHON SOURCE LINES 220-222 Now we will use the following command to get the mappings file. .. GENERATED FROM PYTHON SOURCE LINES 222-225 .. code-block:: Python command = 'sovapply '+source_path + ' '+ bids_path + ' ' + rules_path + ' -m ' + mapping_path print(command) .. rst-class:: sphx-glr-script-out .. code-block:: none sovapply /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/examples/lemon_example_rules.yml -m /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/code/sovabids/mappings.yml .. GENERATED FROM PYTHON SOURCE LINES 226-227 This will produce the following output: .. GENERATED FROM PYTHON SOURCE LINES 227-230 .. code-block:: Python my_output= out(command) print(my_output) .. rst-class:: sphx-glr-script-out .. code-block:: none Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010002/RSEEG/sub-010002.vhdr... Setting channel info structure... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010003/RSEEG/sub-010003.vhdr... Setting channel info structure... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010004/RSEEG/sub-010004.vhdr... Setting channel info structure... .. GENERATED FROM PYTHON SOURCE LINES 231-237 sovaconvert ^^^^^^^^^^^ Now we are ready to perform the conversion given the mapping file just made. Use the following command to print the help of the tool: .. GENERATED FROM PYTHON SOURCE LINES 237-239 .. code-block:: Python command = "sovaconvert --help" print(command) .. rst-class:: sphx-glr-script-out .. code-block:: none sovaconvert --help .. GENERATED FROM PYTHON SOURCE LINES 240-242 This will give the following output .. GENERATED FROM PYTHON SOURCE LINES 242-245 .. code-block:: Python my_output= out(command) print(my_output) .. rst-class:: sphx-glr-script-out .. code-block:: none usage: sovaconvert convert_them [-h] mappings positional arguments: mappings The mapping file of the conversion. optional arguments: -h, --help show this help message and exit .. GENERATED FROM PYTHON SOURCE LINES 246-248 Now we will use the following command to perform the conversion. .. GENERATED FROM PYTHON SOURCE LINES 248-251 .. code-block:: Python command = 'sovaconvert ' + mapping_path print(command) .. rst-class:: sphx-glr-script-out .. code-block:: none sovaconvert /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/code/sovabids/mappings.yml .. GENERATED FROM PYTHON SOURCE LINES 252-253 This will produce the following output: .. GENERATED FROM PYTHON SOURCE LINES 253-256 .. code-block:: Python my_output= out(command) print(my_output) .. rst-class:: sphx-glr-script-out .. code-block:: none Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010002/RSEEG/sub-010002.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/README'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/no USB Connection to actiCAP', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/eeg/sub-010002_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/eeg/sub-010002_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/eeg/sub-010002_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/eeg/sub-010002_task-resting_channels.tsv'... Copying data files to sub-010002_task-resting_eeg.vhdr Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/sub-010002_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/sub-010002_scans.tsv entry with eeg/sub-010002_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010002/eeg/sub-010002_task-resting_eeg.json'... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010003/RSEEG/sub-010003.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/no USB Connection to actiCAP', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/eeg/sub-010003_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/eeg/sub-010003_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/eeg/sub-010003_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/eeg/sub-010003_task-resting_channels.tsv'... Copying data files to sub-010003_task-resting_eeg.vhdr Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/sub-010003_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/sub-010003_scans.tsv entry with eeg/sub-010003_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010003/eeg/sub-010003_task-resting_eeg.json'... Extracting parameters from /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon/sub-010004/RSEEG/sub-010004.vhdr... Setting channel info structure... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/participants.json'... The provided raw data contains annotations, but you did not pass an "event_id" mapping from annotation descriptions to event codes. We will generate arbitrary event codes. To specify custom event codes, please pass "event_id". Used Annotations descriptions: ['Comment/actiCAP Data On', 'New Segment/', 'Stimulus/S 1', 'Stimulus/S200', 'Stimulus/S210'] Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/eeg/sub-010004_task-resting_events.tsv'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/eeg/sub-010004_task-resting_events.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/dataset_description.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/eeg/sub-010004_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/eeg/sub-010004_task-resting_channels.tsv'... Copying data files to sub-010004_task-resting_eeg.vhdr Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/sub-010004_scans.tsv'... Wrote /home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/sub-010004_scans.tsv entry with eeg/sub-010004_task-resting_eeg.vhdr. Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/sub-010004/eeg/sub-010004_task-resting_eeg.json'... Writing '/home/docs/checkouts/readthedocs.org/user_builds/sovabids/checkouts/latest/_data/lemon_bids_cli/dataset_description.json'... .. GENERATED FROM PYTHON SOURCE LINES 257-260 Checking the conversion ^^^^^^^^^^^^^^^^^^^^^^^ For clarity purposes we will check the output directory we got from sovabids. .. GENERATED FROM PYTHON SOURCE LINES 260-267 .. code-block:: Python print_dir_tree(bids_path) print('LEMON CLI CONVERSION FINISHED!') .. rst-class:: sphx-glr-script-out .. code-block:: none |lemon_bids_cli/ |--- README |--- dataset_description.json |--- participants.json |--- participants.tsv |--- code/ |------ sovabids/ |--------- mappings.yml |--------- sovabids.log |--------- sovabids.log.errors |--- sub-010002/ |------ sub-010002_scans.tsv |------ eeg/ |--------- sub-010002_task-resting_channels.tsv |--------- sub-010002_task-resting_eeg.eeg |--------- sub-010002_task-resting_eeg.json |--------- sub-010002_task-resting_eeg.vhdr |--------- sub-010002_task-resting_eeg.vmrk |--------- sub-010002_task-resting_events.json |--------- sub-010002_task-resting_events.tsv |--- sub-010003/ |------ sub-010003_scans.tsv |------ eeg/ |--------- sub-010003_task-resting_channels.tsv |--------- sub-010003_task-resting_eeg.eeg |--------- sub-010003_task-resting_eeg.json |--------- sub-010003_task-resting_eeg.vhdr |--------- sub-010003_task-resting_eeg.vmrk |--------- sub-010003_task-resting_events.json |--------- sub-010003_task-resting_events.tsv |--- sub-010004/ |------ sub-010004_scans.tsv |------ eeg/ |--------- sub-010004_task-resting_channels.tsv |--------- sub-010004_task-resting_eeg.eeg |--------- sub-010004_task-resting_eeg.json |--------- sub-010004_task-resting_eeg.vhdr |--------- sub-010004_task-resting_eeg.vmrk |--------- sub-010004_task-resting_events.json |--------- sub-010004_task-resting_events.tsv LEMON CLI CONVERSION FINISHED! .. rst-class:: sphx-glr-timing **Total running time of the script:** (1 minutes 11.438 seconds) .. _sphx_glr_download_auto_examples_lemon_example.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: lemon_example.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: lemon_example.py ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_