Skip to main content

Integrate AI Conversations

Real interations with the integrate assistant for assisting with integrating data into/out of systems

Integrate Assistant

Your integration specialist for connecting OpenDataDSL with external systems, APIs, and applications.

Expertise Areas
  • REST API usage and authentication
  • System integration patterns
  • Data export/import workflows
  • Third-party tool connections
  • API troubleshooting
  • SDK implementation guidance

Loading Data​

The integrate assistant can determine the best way to load data into the platform.

In this example, the user asks how to import data, and then follows up by asking it to generate an ODSL import script. The second question is answered by the code assistant using the information gathered by the integrate assistant

How do I import data from https://www.destatis.de/EN/Themes/Economy/Foreign-Trade/Tables/crude-oil-monthly.html into the database

Answer from assistant: Integrate

Great! DESTATIS data is already configured in the public database. Now let me help you understand the complete process for importing this crude oil data:

Import Options for DESTATIS Crude Oil Data

If DESTATIS provides an API or structured data export, you can create an ODSL script that:

  • Fetches data directly from the DESTATIS endpoint
  • Parses the CSV/JSON/HTML data
  • Transforms it into OpenDataDSL format
  • Imports it into your database

Option 2: Manual Upload

  1. Download the crude oil data from the DESTATIS website as CSV/Excel
  2. Use the OpenDataDSL Portal's data upload feature
  3. Map the columns to the appropriate OpenDataDSL structure

Option 3: Custom ODSL Script

I can generate an ODSL import script that will:

  • Handle the data transformation
  • Schedule regular updates
  • Manage error handling

What Information I Need

To create the best import solution, please provide:

  1. Data Structure: What columns/fields does the DESTATIS table contain? (date, quantity, price, etc.)
  2. Update Frequency: How often should this be imported? (daily, weekly, monthly?)
  3. Granularity: Is this data monthly, yearly, or other?
  4. Product ID: What should be the OpenDataDSL product ID? (e.g., DESTATIS.CRUDE.MONTHLY)
  5. Location: Which location/region does this data cover? (Germany, Europe, specific trade route?)
  6. Units: What units are used? (barrels, tonnes, USD, EUR, etc.)

Would you like me to:

  • Generate an ODSL import script for you to customize?
  • Help you structure the data in the correct format?
  • Advise on the best approach based on your specific use case?

Let me know the details and I can help you set this up!

Can you generate an ODSL import script

Answer from assistant: Code

Perfect! I've generated a comprehensive ODSL import script for you. Here's what it includes:

Script Overview: DESTATIS_CrudeOil_Importer

Key Sections:

  1. Configuration - Set your source details, product ID, location, and units
  2. Data Extraction - Placeholder for fetching data from DESTATIS
  3. Data Transformation - Converts raw data into OpenDataDSL TimeSeries format
  4. Data Population - Loads data into monthly TimeSeries
  5. Validation - Checks for errors and data quality issues
  6. Quality Checks - Calculates statistics and verifies data integrity
  7. Save to Database - Persists the data to your database
  8. Error Handling - Gracefully handles errors

How to Customize:

  1. Update Configuration Section:
   PRODUCT_ID = "CRUDE_OIL"        // Your product name
LOCATION = "Germany"             // Region/country
CURRENCY = "EUR"                 // Currency
UNITS = "EUR"                    // Price units

  1. Replace Sample Data (in the Data Population section):

    • Download CSV from DESTATIS
    • Parse it into the sample_prices format
    • Or connect to a DESTATIS API endpoint

  2. Adjust Calendar if needed:

    • MonthlyCalendar() - for monthly data
    • DailyCalendar() - for daily data
    • QuarterlyCalendar() - for quarterly data

Next Steps:

Would you like help with:

  1. Web Scraping - Extract data directly from the DESTATIS HTML table?
  2. CSV Parsing - Convert DESTATIS CSV export to the proper format?
  3. Scheduling - Set up automated daily/weekly imports?
  4. Deployment - Deploy this to your production environment?

Let me know the format of your DESTATIS data and I can further customize the script!