06/07-26 MC2 Curation Workflow

[aclayton555]

Discussed with Ashley and Adtiya - June and July to be curated together due to competing priorities and conference attendance in late June. Aim to finalize by end of July (before Sage Week).

Curation and Annotation

• Run Pubmed crawler to generate PublicationView manifest

• Send Amber and Jineta a copy of the PublicationView manifest from latest crawl to review for MC2 Center Newsletter publication highlights (see also https://sagebionetworks.jira.com/browse/CPO-1961) - due July 7

• Send Amber "News from CCKP" for MC2 Center Newsletter (see also https://sagebionetworks.jira.com/browse/CPO-1960) - due July 7

• Annotate publications in PublicationView manifest

• Work through a number of the backlog of old restricted access pubs that have become open access (and list how many have been done out of how many remaining)

• Generate ToolView and DatasetView manifests based on PublicationView manifest

• Run the automated curation workflow to upload publications, datasets and tools [This includes splitting manifests, processing and validating manifests, generating target synapse IDs for upload, schema updates, upload to synapse and (in progress) a validation check for uploads)

• Generate UNION tables

• QC of staging tables

• Performing automate portal sync to CCKP

• Validate data on the CCKP

Status check [Plan to report numbers for each category following pubmed crawl]:

• Publication upload

• Dataset Upload

• Tool Upload

[aclayton555]

@aditya-nath-sage just checking - are we still on track to sync the June + July crawls/curation batches before Sage Week (i.e this week since I know you are OOO)?

[aditya-nath-sage]

Have completed curation of pubs datasets and tools and upload/sync of publications!
pubs: https://docs.google.com/spreadsheets/d/1tMwVmlbCLK42bTomEt3G7JBHTe1WT4Jmir02pTM6kkA/edit?usp=sharing
datasets: https://docs.google.com/spreadsheets/d/17FjEQr4Wi9-oYbgn7WwOawIUnNAkQKQHunjn4bW_lE4/edit?usp=sharing

Need to make slight changes to the dataset process (what we discussed in the meeting on 7/24 about dataset entities being found on the table), so holding off the sync on datasets