GitHub - SoftwareCountry/Perseus

Introduction

Perseus combines intuitive and easy to use Web-based UI for design and implement ETL (extract, transform, and load) configuration and service for conversion the native/raw data to the OMOP Common Data Model (CDM).

Additionally, Perseus has embedded tools for search in the standardized vocabularies, generates documentation for the ETL process, create the code mappings and data quality check.

Wiki

Getting started

Contact Us: perseus.support@softwarecountry.com

Features

Map source data to tables and columns to CDM
Combine source tables
Use pre-built sql functions (replace, concat…)
Use pre-built source to source and source to standard vocabulary lookups (icd9, icd10, ndc…)
Create custom lookups for the concept_id fields
Set constant values to the CDM fields
Use System/Auto generated values for the CDM fields
Auto mapping for similar fields
OMOP Vocabulary search
Data Quality check
Search mapping between new codes and OMOP standard concepts
Convert data from native format to CDM
Logic for creating eras (DRUG_ERAs, CONDITION_ERAs…)
Logic for grouping visit occurrence/observation_period records
Auto domain switching
Create ETL specification

Screenshots

Technology

Angular 12
Python 3.7
Java 17
R 4.1.3
PostgreSQL 13.2
.NET Core 3.1

Deployment server requirements

Unix / Windows OS, Docker,
4GB RAM,
~10 GB HDD (Depend on Vocabulary size),
Open ports: 443, 80.

Getting Started

Vocabulary

Get the link to the vocabulary from Athena.

Install vocabulary archive and extract to vocabulary directory. Full path vocabulary-db/vocabulary. For example:

unzip <downloaded archive> -d vocabulary

Database deployment can take a long time if the dictionary size is large enough.

To Docker Compose.

SMTP server

Multi-user

(Optional)

To get user registration links by e-mail you should configure SMTP server settings first. Edit file named user-envs.txt in the user directory with the following content (without spaces):

SMTP_SERVER=<your SMTP server host address>
SMTP_PORT=<your SMTP port>
SMTP_EMAIL=<email from which registration links will be sent to users>
SMTP_USER=<SMTP login>
SMTP_PWD=<SMPT password>
TOKEN_SECRET_KEY=<token encoding key>

to Docker Compose

Test user

Single-user

If you want to skip multi-user mode use user with these credential:

Email:

perseus@softwarecountry.com

Password:

Starting with Docker Compose

To start all containers at once using docker-compose please

make sure docker-compose is installed
set vocabulary link, see Vocabulary section
configure SMTP server as it described in SMTP section (Optional)

Unix:

Windows:

Open localhost:80 in your browser, preferably Google Chrome

Starting each container separately

CONTAINERS

Perseus uses auxiliary services to scan, convert and validate data.

Below are links to these services, which should be included in the app build.

Finally

Open localhost:80 in your browser, preferably Google Chrome.

Getting Involved

User guide and Help: Perseus documentation
We use the: GitHub issue tracker
Contact us: Email

License

Perseus is licensed under Apache License 2.0