Jailer: A tool for database subsetting, schema and data browsing

Hacker News - Fri Jan 14 20:23

Jailer is a tool for database subsetting, schema and data browsing. It exports consistent, referentially intact row-sets from relational databases. It removes obsolete data without violating integrity. It is DBMS agnostic (by using JDBC), platform independent, and generates DbUnit datasets, hierarchically structured XML, and topologically sorted SQL-DML.

Features

  • Exports consistent and referentially intact row-sets from your productive database and imports the data into your development and test environment.
  • Improves database performance by removing and archiving obsolete data without violating integrity.
  • Generates hierarchically structured XML, topologically sorted SQL-DML and DbUnit datasets.
  • A demo database is included with which you can get a first impression without any configuration effort.

News
2022-01-01   Comprehensive redesign and modernization of the entire user interface. New Look & Feel FlatLaf.
2021-02-04   Cycles in parent-child relationships will be detected and broken. Thus, such data can be exported by deferring the insertion of nullable foreign keys.
2020-02-04   The Jailer engine is published in Maven repository .
2019-02-01   The new "Model Migration Tool" allows you to easily find and edit the newly added associations if the data model has been extended after the last change to this extraction model.
2018-04-26   The new feature "Analyze SQL" analyzes SQL statements and proposes association definitions. This allows to reverse-engineer the data model based on existing SQL queries.
2018-03-06   SQL Console with code completion, syntax highlighting and database metadata visualization.
2017-05-10   New API provides programmatic access to the data export and import functionality.
2017-03-30   Improved filter management . Filter Templates allows you to define rules for assigning filters to columns.
Filters on primary key columns will automatically be propagated to the corresponding foreign key columns.
2015-12-04   Data can now also be exported directly to a schema of the same database. This ensures optimal performance.
2015-10-23   Rows can alternatively be collected in a separate embedded database. This allows exporting data from read-only databases.
2014-07-20   Implemented the "Subset by Example" feature: Use the Data Browser to collect all the rows to be extracted and let Jailer create a model for that subset.
2014-04-15   A Data Browser  Videos  has been introduced. Navigate bidirectionally through the database by following foreign-key-based or user-defined relationships.

About Jailer

Databases are growing in both size and complexity to meet the increasing demands of business. Applications to process the data are also increasing in size and complexity. With the growing complexity, solid testing becomes more and more important in order to assure the quality of software. Ideally we would like to test all changes against up-to-date production data, so the general practice is use a copy of the production database for all testing.

But when a database exceeds a certain size it becomes very expensive to provide full-size copies of the production database for development and testing. One solution of this problem is to have fewer full size copies of the production database than are really needed, often only one, which will be shared between the development and testing teams.

Of course this is far from optimal. Data in the database is left in an unknown state when passed from one team to the other. It takes a long time to provide a refresh of the production copy when it's required. Always having an up-to-date production copy is almost impossible.

The databases required for development and testing rarely need to be full size, it is often easier to work on a small copy. Unfortunately it is very hard to manually extract a small subset of the production data. It is not possible to just take 10% of each table to get a 10% size database. The data in one table would not be related to the data in the other tables. It would not be referentially intact.

Jailer simplifies the extraction of referentially intact data. Once you have defined an extraction model, it can be used to extract data from the production database fast and easy whenever up-to-date test data is required.