QUSMA Data Management System – QDMS (on github)
The QUSMA Data Management System (QDMS) is a client/server system for acquiring, managing, and distributing low-frequency historical and real-time data, written in C#.
The server acts as a broker between clients and external data sources, as well as a local database of historical data. The server UI allows its use without the need for a client application. Here’s a rough view of how the systems are connected to each other.
A client library is provided which can access the server either locally or over a network, to request data, metadata, etc. A simple sample application showing usage of the client can be found here.
QDMS uses ZeroMQ and Protocol Buffers for client/server communications, MahApps.Metro for the interface, and ib-csharp to communicate with IB’s TWS.
If you wish to contribute, fork the repo and send a pull request with your changes.
For bug reports, feature requests, and general discussion please use the github issues.
Click here to install.
Features:
- Manage metadata on stocks, options, futures, CFDs, etc.
- Download historical and real time data from external data sources.
- Local storage of historical data.
- Continuous futures data.
- Schedule automatic data updates.
- CSV import/export.
Requirements:
- MySQL, MariaDB, or SQL Server
Currently Supported Data Sources:
- Yahoo
- Interactive Brokers
- Quandl
- FRED (Federal Reserve Economic Data)
- Google Finance
You can find the source code on GitHub: https://github.com/qusma/qdms.
Screenshots:
Instrument metadata.
Adding a new instrument from Interactive Brokers.
Importing data from a CSV file.
Main server GUI screen.
adrianwho says:
I saw qdms supports 3 data sources (yahoo finance, interactive brokers and quandl). I am just starting using data sets, can you tell me, as per your experience, data from quandl (which I just heard of, and looks interesting) is accurate? What about data from yahoo finance? And for the moment I am more interested in stock prices. As I found in some forums, yahoo finance data may have some wrong values.
qusma says:
Quandl generally has low-quality data (tons of missing data), but it depends on what the source is. The advantage is the extremely wide array of data sets they have.
Yahoo sources its data from CSI, it’s generally very good. A couple of times I’ve seen some obviously wrong values but generally I think it’s trustworthy.
Divergences: Yield Spreads and Size : QUSMA says:
[…] (the QUSMA Data Management System) as it’s getting more mature. You can find the link on the project’s page. It’ll prompt you for an update when a new version comes […]
Announcing QPAS: Open Source Performance, Risk, and Execution Analytics : QUSMA says:
[…] additional data for things like charting, execution analysis, and benchmarking. By default it uses QDMS, but you can use your own data source by implementing the IExternalDataSource […]
Mark says:
hi there,
when would you think something like IQFeed or BarChart could be supported as a data-source?
thx
qusma says:
I have dropped my IQFeed subscription so unless someone else writes it, it’s not happening any time soon. Also they charge for access to their API so I suspect the license might not allow open source projects to use it at all.
Mark says:
BarChart offers for free (“trial”) a limited number of various symbols (stocks, futures, indexes etc) per day. Beauty of it is this trial also has tick data (a week’s worth if i remember).
OEC accounts also offer decent data on futures.
Let me know if such integration is possible.
qusma says:
If you know C# it’s not that hard to implement another data source, you need to write a class that implements IHistoricalDataSource (https://github.com/qusma/qdms/blob/master/QDMS/Interfaces/IHistoricalDataSource.cs ) and IRealTimeDataSource (https://github.com/qusma/qdms/blob/master/QDMS/Interfaces/IRealTimeDataSource.cs ).
The current implementations will probably help, you can find them here: https://github.com/qusma/qdms/tree/master/QDMSServer/DataSources
Mark says:
If you could get access to their API interface & docs temporarily
and build such an interface – then I assume that piece of code can be
added as a binary resource (dll/assembly,etc) into the project without
any problems. Isn’t it ?
qusma says:
That should be doable, yeah. As long as the dev license isn’t time-limited of course, I’m not sure how that works.
Now that I think about it, it might even be possible to use an iqfeed connector dll from another program (like amibroker or multicharts). It would be illegal to distribute it obviously, but one could side-step that issue by implementing a plugin system for the datasources.
Mark says:
Asked, and there’s an annual fee to have API access … I don’t know how MC/AmiBr or the other “big boys” are doing it. Do you own a MC.NET license? I figure it’s not too difficult to programmatically access an external database from their .NET environment, and say for example, save the data, while connected to IQFeed.