Go Back
Print the Schema of a Connector or Table
Publisher
: Lingk
Run In Lingk
Description
This recipe convert data from a connector to a schema. Schemas are important because they enable you to use NULL columns in a queries without errors.
Browse the knowledge base
Twitter
E-Mail
# _____ _ _____ __ # | __ \ (_) |_ _| / _| # | |__) |___ ___ _ _ __ ___ | | _ __ | |_ ___ # | _ // _ \/ __| | '_ \ / _ \ | | | '_ \| _/ _ \ # | | \ \ __/ (__| | |_) | __/ _| |_| | | | || (_) | # |_| \_\___|\___|_| .__/ \___| |_____|_| |_|_| \___/ # | | # |_| # Project Name - PRINT THE SCHEMA OF A CONNECTOR OR TABLE # Recipe URL - https://app.lingk.io/a/10932/tf/17885 # Description - # This recipe convert data from a connector to a schema. # Schemas are important because they enable you to use NULL columns in a queries without errors. # To use this recipe, click Run! # Industry - Higher Ed # Business Process - Graduate Reporting # Systems - # Connectors - JSON # Data Flows - Single Direction # Connection Type - JSON # Add Recipe notes / Change log information here! # _____ _ # / ____| | | # | | ___ _ __ _ __ ___ ___| |_ ___ _ __ ___ # | | / _ \| '_ \| '_ \ / _ \/ __| __/ _ \| '__/ __| # | |___| (_) | | | | | | | __/ (__| || (_) | | \__ \ # \_____\___/|_| |_|_| |_|\___|\___|\__\___/|_| |___/ # # CONNECTORS specify what data will be pulled into the in-memory database during processing connectors: # JSON Setup - https://help.lingk.io/en/articles/74-json-connector-reference ###### Start: JSON Connectors ####### - name: sourceLeadData type: json schemaRef: sourceLeadDataSchema # uncomment after getting the schema properties: jsonObject: > [ { "leadId": "1", "firstName": "John", "lastName": "Thomas", "leadScore": 88 }, { "leadId": "2", "firstName": "Anne", "lastName": "Jacobs", "leadScore": 32 } ] schemas: # paste schema from "printschema" statement here - name: sourceLeadDataSchema fields: - name: 'firstName' type: 'string' - name: 'lastName' type: 'string' - name: 'leadId' type: 'string' - name: 'leadScore' type: 'long' - name: 'nullColumn' # this is a manually added schema for a field that is used in a query but not available in the test dataset type: 'string' ###### End: JSON Connectors ####### # _____ _ _ _ # / ____| | | | | | # | (___ | |_ __ _| |_ ___ _ __ ___ ___ _ __ | |_ ___ # \___ \| __/ _` | __/ _ \ '_ ` _ \ / _ \ '_ \| __/ __| # ____) | || (_| | || __/ | | | | | __/ | | | |_\__ \ # |_____/ \__\__,_|\__\___|_| |_| |_|\___|_| |_|\__|___/ # STATEMENTS specify how the data should be processed while in memory statements: #******************************************************************** D I S C L A I M E R *********************************************************************************************** # * # Note that in an effort to keep recipes optimized for DPH (Data Processing Hours), print statements should be commented out after development has concluded for a recipe. * # For more information on DPH optimization, please visit the following help article - https://help.lingk.io/en/articles/212-minimizing-data-processing-hours-on-the-lingk-platform * # * #******************************************************************** D I S C L A I M E R *********************************************************************************************** # PRINT the schema of an connector or table # - statement: PRINTSCHEMA sourceLeadData #- statement: printschema sourceLeadData # NOTE: You can add trasformations into your statements - statement: | (transformedData) => SELECT leadId lead_id, ucase(firstName) first_name, lastName last_name, leadScore lead_score FROM sourceLeadData #- statement: PRINT transformedData - statement: | (transformedData) => SELECT leadId lead_id, ucase(firstName) first_name, lastName last_name, leadScore lead_score, nullColumn FROM sourceLeadData #- statement: PRINT transformedData # count the data - statement: (dataCount) => SELECT count(*) FROM transformedData #- statement: PRINT dataCount # Add more statements to convert, join, aggregrate, transform, and integrate your data
Automatic Union of Multiple JSON files
Join JSON and database data to output an Excel Spreadsheet