Skip to main content

Accessing SureChEMBL data in bulk




It is the peak of the summer (at least in this hemisphere) and many of our readers/users will be on holiday, perhaps on an island enjoying the sea. Luckily, for the rest of us there is still the 'sea' of SureChEMBL data that awaits to be enjoyed and explored for hidden 'treasures' (let me know if I pushed this analogy too far). See here and here for a reminder of SureChEMBL is and what it does. 

This wealth of (big) data can be accessed via the SureChEMBL interface, where users can submit quite sophisticated and granular queries by combining: i) Lucene fields against full-text and bibliographic metadata and ii) advanced structure query features against the annotated compound corpus. Examples of such queries will be the topic of a future post. Once the search results are back, users can browse through and export the chemistry from the patent(s) of interest. In addition to this functionality, we've been receiving user requests for local (behind the firewall), batch access of the SureChEMBL data almost since day one of releasing the interface last year. By such data, we mean not only the compound repository (which is available anyway) but also the comprehensive map of associations between compound and patents, i.e. which compound was extracted from which section of which patent, for all available compounds and patents. To address these requests, we currently provide two additional ways to access the SureChEMBL data:

1) The map file. This is a flat file including compound, patent and association information. The back file contains all associations for eligible patent documents published between 1960 and 2014. Since then, two more incremental update files have been available covering the first 2 quarters of 2015. More info on the structure of the file is found here and all files are available here. We will continue to add incremental updates quarterly. 

2) The data client. This is a script that firstly creates a relational database in the user's environment and then populates the database with SureChEMBL data it retrieves from our private ftp server on a nightly basis. In addition to the front file (the patent and compound info that arrives daily), the script can also fetch the back file for each past year and back-fill the database accordingly. In the end, the data client script effectively creates a local and live snapshot of the SureChEMBL database, which allows for further integration, querying, filtering, deployment, etc., on the users' side. More info on the details of the client can be found here and documentation on how to set the client up is available here. Note that the script will need credentials to access our private ftp server, which we are happy to provide if you ask. It is also worth mentioning that the data client provides more information in a more timely fashion than the map file (daily vs. quarterly) but it requires considerably more effort and resources to set it up, compared to reading a flat file. 

As usual, this is not all - there is soon going to be an additional way to batch-access the SureChEMBL data. As announced recently, an enhanced version of SureChEMBL data (chemical AND biological annotations from patents), will be semantically integrated in the OpenPHACTS system and will be part of the freely available OpenPHACTS API. This will be available later this year.


George

Comments

Popular posts from this blog

Target predictions in the browser with RDKit MinimalLib (JS) and ONNX.js

Some time ago we showed an example of how a model trained in Python's PyTorch could be run in a C++ backend by exporting it to the ONNX format.  Greg also showed us in his blogpost how our multitask neural network model could be used in a very nice KNIME workflow by exporting it to ONNX. That was possible thanks to RDKit's Java bindings and the ONNX Java runtime. As a refresher, most of the most popular machine learning frameworks can export their models to this format and many programming languages can load them to run the predictions. This certainly is a beautiful example of interoperability! In November 2019 RDKit introduced a reduced functionality Javascript library which is able to do all we need in order to use our multitask model in the browser. So, the only thing that was left to do was to combine these two awesome tools... and we did it! Here is our demo with its available source code . Start typing a smiles into the box and enjoy! Updated code to generate the m

ようこそ、ケンブルへ! - Welcome to 剣舞瑠 ! -

The following is written in Japanese.... ケンブルチーム(ChEMBL Team)は、欧州バイオインフォマティクス研究所( EMBL-EBI )にあり、創薬研究に有用な化合物やターゲット情報を提供するデータベースを開発しています。 ChEMBLdb は、創薬研究に有用な医薬品化合物の情報を提供するデータベースです。現在、約50万個の化合物情報、約190万件の活性情報及びそれらのターゲット情報が登録されています。ユーザーは、生物活性化合物の情報を部分構造検索や類似性検索で調査したり、また、ターゲットのアミノ酸配列からBLAST検索でアッセイ情報を収集することができます。 ケンブルチームでは、キナーゼに特化したカイネースサファリ( Kinase SARfari )のサービスも開始しました。 日本語でのご質問、ご要望は kaz(at)ebi.ac.uk までどうぞ。チームメンバー一同、皆さんのご利用をお待ちしています!

This Python InChI Key resolver will blow your mind

This scientific clickbait title introduces our promised blog post about the integration of UniChem into our ChEMBL python client. UniChem is a very important resource, as it contains information about 134 million (and counting) unique compound structures and cross references between various chemistry resources. Since UniChem is developed in-house and provides its own web services , we thought it would make sense to integrate it with our python client library . Before we present a systematic translation between raw HTTP calls described in the UniChem API documentation and client calls, let us provide some preliminary information: In order to install the client, you should use pip : pip install -U chembl_webresource_client Once you have it installed, you can import the unichem module: from chembl_webresource_client.unichem import unichem_client as unichem OK, so how to resolve an InChI Key to InChI string? It's very simple: Of course in order to reso

Mini-project: Training a DNN in Python and exporting it to the ONNX format to run its predictions in a C++ micro-service

Python is nowadays the favourite platform for many machine learning scientists. It is an easy to learn language that provides a vast number of nice data science and AI tools perfect for rapid prototyping. Models trained using Python DNN libraries like PyTorch and Tensorflow usually perform well enough to be used for production runs, but there are some situations that require the predictions to be run in C++ i.e., when best performance is required or when the model needs to be integrated with an existing C++ codebase. ONNX (Open Neural Network Exchange) is an AI framework designed to allow interoperability between ML/DL frameworks. It allows, for example, models trained in scikit-learn, PyTorch, TensorFlow and other popular frameworks to be converted to the "standard" ONNX format for later use in any programming language with an existing ONNX runtime. In this mini-project we trained a "dummy" DNN network (single task target predictor) in Python with PyTo

A python client for accessing ChEMBL web services

Motivation The CheMBL Web Services provide simple reliable programmatic access to the data stored in ChEMBL database. RESTful API approaches are quite easy to master in most languages but still require writing a few lines of code. Additionally, it can be a challenging task to write a nontrivial application using REST without any examples. These factors were the motivation for us to write a small client library for accessing web services from Python. Why Python? We choose this language because Python has become extremely popular (and still growing in use) in scientific applications; there are several Open Source chemical toolkits available in this language, and so the wealth of ChEMBL resources and functionality of those toolkits can be easily combined. Moreover, Python is a very web-friendly language and we wanted to show how easy complex resource acquisition can be expressed in Python. Reinventing the wheel? There are already some libraries providing access to ChEMBL d