ChEMBL Resources

Resources:
ChEMBL
|
SureChEMBL
|
ChEMBL-NTD
|
ChEMBL-Malaria
|
The SARfaris: GPCR, Kinase, ADME
|
UniChem
|
DrugEBIlity
|
ECBD

Friday, 29 May 2015

Compound popularity contest


Have you ever wondered which compound is the most popular in ChEMBL? And by popular I don't mean the one which cracks the best jokes at dinner parties; I mean the compound with the largest number of structural analogues or nearest neighbours (NNs). This number also gives an indication of the sparsity or density of the chemical space around a compound and is a useful concept during hit expansion and lead optimisation. 

This number of course depends on the fingerprint, the hashing and folding parameters, the similarity coefficient and the threshold. So let's say 2048-bit RDKit Morgan fingerprints with a radius of 2 or 3 (equivalent to ECFP_4 or ECFP_6) and Tanimoto threshold of 0.5. Why so low threshold? For an explanation, see here and here.

To calculate this compound 'popularity', one would need to calculate the full similarity matrix of the 1.4M compounds in ChEMBL. This used to be prohibitively computationally expensive just a few years ago; now, thanks to chemfp, it only takes 5 commands and a few hours to calculate the matrix and counts at a certain similarity threshold on a decent machine. Here's how it's done:

#install chemfp
pip install chemfp

#get the chembl sdf

#calculate rdkit morgan fps
rdkit2fps --morgan --radius 3 --id-tag "chembl_id" --errors report chembl_20.sdf.gz -o rdkit_chembl.fps

#calculate counts of neighbours at a given threshold, here 0.5
simsearch --threshold 0.5 --NxN -c rdkit_chembl.fps -o chembl_sim_matrix.txt

#sort by number of neighbours and find most 'popular' compound

sort -rn chembl_sim_matrix.txt | head -25


Here's a list of the most popular ones with the number of NNs:

1570 CHEMBL3037891
1540 CHEMBL2373012
1523 CHEMBL3086861
1461 CHEMBL2158309
1451 CHEMBL440810
1451 CHEMBL414360
1428 CHEMBL2401865
1425 CHEMBL408133
1425 CHEMBL354100
1425 CHEMBL344931
1425 CHEMBL310737
1425 CHEMBL303256
1425 CHEMBL302290
1425 CHEMBL27867
1425 CHEMBL137783
1403 CHEMBL1866472
1396 CHEMBL3102922
1392 CHEMBL3102921
1392 CHEMBL3102920
1387 CHEMBL91573
1387 CHEMBL70362
1387 CHEMBL601767
1387 CHEMBL533732
1387 CHEMBL503883
1387 CHEMBL477

Most of the most popular compounds are large, usually peptides, e.g. CHEMBL3037891. We'll leave it as an exercise for the reader why that is. But this not always the case, for example CHEMBL477 is adenosine and it genuinely has a lot of near neighbours in ChEMBL (including a lot of stereo variants).  

And what about the least popular compounds, i.e. the ones with no neighbours above 0.5 Tanimoto? One would not expect a lot of them, since ChEMBL by design contains compounds from congeneric series as reported in the med. chem. literature. And yet, due to small size, symmetry (CHEMBL100050), possible mistakes and peculiarities of the fingerprint, or due to being genuinely lonely (CHEMBL35416), they do exist.  

Apart from popularity contests, a similarity/distance matrix of ChEMBL compounds is the first step for clustering and graph/community detection analysis.


George


PS: Thebacon has only 22 neighbours...

Friday, 15 May 2015

What's Going On?



I’ve been asked a lot by mail recently ‘What’s Going On?’ Well, here is are some facts and some emotion.

So today is my last day at work here at EMBL-EBI. It’s been a fun and thrilling ride (for me at least), I’ve made lots of new friends, living life as an Open Data advocate and academic researcher, and most importantly having the privilege to lead the team here responsible for the ChEMBL database. It had been a long-term goal of mine to unlock large-scale bioactivity data from proprietary data silos and eye-wateringly expensive paywalls; so as US President George Bush famously said ‘Mission Accomplished!’. The impact of ChEMBL on academia, SMEs and large pharma has been great - and you can see the impact in new method development, but more importantly in potential new future drugs. My personal indebtedness to the Wellcome Trust for their support is immeasurable. An additional big shout out to Digital Science for their vision in donating the SureChEMBL platform to us.

I’ll be starting a new blog, covering my next adventure - artificial intelligence-enhanced drug discovery. I’ll tweet when this is up and running, but the first few weeks at a new job, as I’m sure you know, is spent sorting out pencils, working out where the best coffee is hidden and most importantly navigating the office politics of the milk in the fridge. When this blog starts up, I’ll tweet the url. For those of you interested in the ChEMBL groups activities, make sure you follow @ChEMBL and @SureChEMBL. If you want to see what I'm up to next, it's at @StratMed.

If any of you are ever in the West End of London (which to non-native Londoners actually means the centre) get in touch with me, and I’ll try to treat you to an orange mocha frappuccino.

Now for the bit you all actually care about….

  • Anne Hersey is taking over the ChEMBL Wellcome Trust Strategic Award grant for ChEMBL (which also covers SureChEMBL). Many of you will know Anne already, and know just what good news this is. Anne is also taking over the majority of our other grants and activities of the group, including our participation in IMI eTox, NIH IDG KMC, & GSK CTTV grants.
  • Jo McEntyre will be PI for EMBL-EBI on the IMI OpenPHACTS grant, although the majority of the work will be done by ChEMBL group staff. If you don’t know about the Open PHACTS platform, check out what they have done!
  • Ugis Sarkans will become PI for EMBL-EBI on the FP7 HeCaTos grant. The data content and modelling components will be done by ChEMBL group staff.


If you have general questions about ChEMBL or SureChEMBL first try the support email addresses chembl-help@ebi.ac.uk and surechembl-help@ebi.ac.uk.


Thursday, 7 May 2015

The Ying and Yang of Drugs and Targets


Today is election day in the UK, and so it's ones duty to vote, and then spend the rest of the day thinking about just how cool science is.

I was thinking the other day on the train, that we don't always want drugs that are benign, of course we always want drugs that are safe and predictable in action; but mechanistically we sometimes want to do harm, kill a cell, wipe it out entirely - let's call these malign. Of course, this socially responsible expression of this desire to obliterate life with drugs is in the development of antibiotics (bacteria, viruses, fungi and parasites) and also anti-cancer agents. The other system we sometimes want to modulate to cause specific harm is via activation of the immune system - priming it to attack non-self (pathogen) cells, or that limbo-land of pseudo-self cancer cells.

Anyway, this thinking of drugs as either (simplistically) malign or benign was triggered by a mail from a new collaborator interested in antiparasite agents identified via drug repositioning. How could you come up with a prioritised and annotated set of current drugs (and their associated cognate targets) that are biased to killing certain cells?


There are many approaches to this problem, here is just one... Given that the majority of drugs try to do no harm, and restore normal function to the recipient, screening most 'benign' drugs would be expected a priori to do little. Do we really want to find a way to balance sugar metabolism in a plasmodium and cure it's diabetes? - of course not, so screening known antidiabetic agents is less likely to reveal a cidal phenotype.
  • sporadic binding of a non-cidal drug/target to a new cidal target in the pathogen.
  • a benign-malign target switch occurs - so a context (organism) dependent switch happens - a nice example here is the statins, which were first isolated and characterised as anti fungal natural products, and now of course are used as cholesterol lowering agents in humans. It’s left as an exercise for the reader to answer why statins aren’t used as antifungals.


There are exceptions to all such half-assed approximations. The two most obvious are

So, let’s have a high level go at classifying drugs as malign or benign in intent. I’m on a train, so it will need to be a quick and dirty approach…. Use the WHO ATC classification! Classes J, L and P ('Antiinfectives for systemic use', 'antineoplastic and immunomodulating agents', and 'anti-parasitic products, insecticides and repellents' respectively) are clearly enriched in malign drugs (and malign targets), and certain subclasses of other top level classes are also malign - e.g. D01, D06 and G01 for example. 

So, we can rapidly (and approximately) classify drugs into malign and benign classes using the ATC, and via simple extension produce sets of malign and benign targets. We can even address some of the more-likely benign to malign switches, for example using sequence similarity to identify similar architectures/mechanisms. As an example, quite a few antithelmintics work by binding to ion channels and causing paralysis, finding ATC classes of ion-channel blockers is straightforward.



Toodlepip, for ever, jpo!

Thursday, 30 April 2015

Easter egg hunt results.



As promised, we would like to provide the answer to the Easter egg hunt competition and announce the winner. Exactly seven hours after publishing the blog post we received a comment with the correct answer. The author of the comment was Matt Swain, who runs his blog about cheminformatics.

You can verify the correctness of his answer by visiting the password protected link in the original post. The password is:

fu']zxp+Wm[Kc3-N

Congratulations to Matt!

ChEMBL team

Thursday, 9 April 2015

Upcoming Webinars




We are pleased to announce a new round of resource-specific webinars that will be given in May and June 2015. These four webinars will cover UniChem, ChEMBL, MyChEMBL (the ChEMBL Virtual Machine) and the ChEMBL Web Services.


  • ChEMBL Web Services, 4pm BST, 17th June 2015,   Currently Postponed


For those of you who can't make these days/times, each 1 hour long webinar will be video recorded and will be available to watch on YouTube afterwards. Additionally, we will make the slides available for download.


The video for last month's SureChEMBL webinar can be found here (and part 2 here). 


For more information about the webinars, or to suggest other topics to cover, please contact chembl-help@ebi.ac.uk.

Wednesday, 1 April 2015

Easter egg hunt


Easter is coming and for all those, who don't know what to do with their spare time and fancy entering a little competition, we've prepared a small challenge.

Easter Egg?

In software development, an Easter egg is funny (but harmless) and undocumented feature hidden from users in unusual places. Excel 97 has its Flight Simulator, FireFox about:robots address and Debian's apt-get has a moo command. The ChEMBL web services has now joined this list and we invite you to find its hidden feature and share with others.

But why?

We would like to encourage you to look at the source code of our web services.  Reading code is essential developer skill, as it helps in understanding how the code works. This can lead to the development of new software and/or improve an existing codebase. After skimming through the code, hopefully you will agree that it is well written and easy to extend. Let us know if you disagree, either by emailing us or creating a GitHub issue. We promise, there are no dragons there, only an Easter egg, which should be easy to find after reading the code.

How?

The first person to comment on this post with a URL revealing the Easter egg (web service URL, hmm hint maybe?), will be honoured with a mention in a future blog post. As a 'guarantee', that the Easter egg exists now, we provide the following password protected URL - we will reveal the password, the winner and more details in the follow up blog post. Do you think you can do it? Check it out!

Is this an April fool joke?

No. The joke is in having an Easter egg, not in finding it.

Monday, 23 March 2015

Compound names and The InterWeb


We have a long-standing interest in finding clinical stage compounds from the literature - and it turns out that the peer reviewed literature is pretty useless, by the time something is published and appears in print it is old news, and although reviews of particular areas or targets are useful in capturing a snapshot, they are not really useful in decision support - using data to inform future experiments and investment. So these things need to be databased and online to be of much use.

So, we have a pretty big set of clinical stage kinase inhibitors that we've gathered from a wide variety of sources - this is the subject of a paper we're currently writing up, so I won't bore you with how we got the data, well, not right now.

We've posted a couple of times before about the transition of names, or classes of names as compounds go through development and approval - a search of the ChEMBL-og will show you these. But here's something hot off the press.

Each row is a distinct kinase inhibitor, each column is a synonym or identifier for that compound. Columns 1 to 5 are research code type names (e.g. UK-92480), with the first column being the one we use as the primary identifier, Column 6 is the InChI key, Column 6.1 is the CAS Registry number, 7 is the USAN/INN 8 is a deprecated USAN/INN or another common trivial name, Columns 9 to 12 are trade names (in case you are wondering, row 12 is of the Chinese tradename of a compound). The cells are coloured by the log(10) of the count of the number of times the name occurs in a google search - pink high, blue low (or there is no synonym of that class for that compounds). There are some false positives, where the name is unusually common, so it matches the name of something unrelated to a kinase inhibitor or therapeutic application.

It's interesting to see the diversity of names increasing as a compound becomes a launched drug, but also the broad coverage for many of these compounds with hits to the InChI key - and also for what fraction compound structures are known.

The most frequent hits in google are to:

ALL-3  11000000
CIF     7400000
X-82   1930000
AT-877  623000
X-396   425000
DE-10  362000
AC-220   245000
R-112     226000
Imatinib  226000
Erlotinib  148000

Note this is without any filtering, or other fancy stuff, just raw search engine hit counts.


Francis and jpo

Friday, 20 March 2015

The SureChEMBL map file is out


As many of you know, SureChEMBL taps into the wealth of knowledge hidden in the patent documents. More specifically, SureChEMBL extracts and indexes chemistry from the full-text patent corpus (EPO, WIPO and USPTO; JPO titles and abstracts only) by means of automated text- and image-mining, on a daily basis. We have recently hosted a webinar about it which turned out to be very popular - for those who missed it, the video and slides are here.

Besides the interface, SureChEMBL compound data can be accessed in various ways, such as UniChem and PubChem. The full compound dump is also available as a flat file download from our ftp server.

Since the release of the SureChEMBL interface last September, we have received numerous requests for a way to access compound and patent data in a batch way. Typical use-cases would include retrieving all compounds for a list of patent IDs, or vice versa, retrieving all patents where one or more compounds have been extracted from. As a result, we have now produced this so-called map file which connects SureChEMBL compounds and patents.

It is available here.
More information can be found in the README file.

What is this file?

There is a total of 216,892,266 rows in the map, indicating a compound extracted from a specific section of a specific patent document. The format of the file is quite simple: it contains compound information (SCHEMBL ID, SMILES, InChI Key, corpus frequency), patent information (patent ID and publication data), and finally location information, such as the field ID and frequency. The field ID indicates the specific section in the patent where the compound was extracted from (1:Description, 2:Claims, 3:Abstract, 4:Title, 5:Image, 6:MOL attachment). The frequency is the number of times the compound was found in a given section of a given patent. More information on the format of the file in the README file.

How many compounds and patents are there?

There are 187,958,584 unique patent-compound pairs, involving 14,076,090 unique compound IDs extracted from 3,585,233 EP, JP, WO and US patent documents - an average of ~52 compounds per patent. The patent coverage is from 1960 to 31-12-2014 inclusive.

Here's a breakdown of the patents in the map per year and patent authority:




Are these all the compounds and patents in SureChEMBL?

Technically, no - in practice, yes. We excluded chemically annotated patents that are not immediately relevant to life sciences, such as this one. For the filtering, we used a list of relevant IPCR and related patent classification codes. At the same time, we excluded too small, too large, too trivial compounds, along with non-organic and radical/fragment compounds.

Are these compounds genuinely claimed as novel in their respective patents?

Automated methods to assess which are the important and relevant compounds in a pharmaceutical patent is a field of research and one of our future plans. For now, the map file include all extracted chemistry mentioned in all sections of a patent, subject to the filters listed in the previous section. A quick and effective trick to filter out trivial and/or uninformative compounds is to use the corpus frequency column and exclude everything with a value more than, say, 1000. Note that, in this way, you will also exclude drug compounds such as sildenafil, which are casually mentioned in a lot of patents. You could also look for compounds mentioned only in claims, description or images sections by filtering by the corresponding field ID.

What can I do with this?

Well, you can start by 'grepping' for one or more patent IDs or SCHEMBL IDs or InChI keys, followed by further filtering. Many of you will choose to normalise the flat file into 3 database tables (say compounds, documents and doc_to_compound) for centralised access and easy querying.

For example, to find the patents the drug palbociclib has been extracted from:

Any plans to update this map file?  

New patents and chemistry arrive and are stored to SureChEMBL every day. We are planning to release new versions and incremental updates of the map file every quarter, in sync with the update of the compound dump files.

I couldn’t find my compound / patent - this compound should not be there

Don’t forget this an automated, live, high-throughput text-mining effort against an inherently noisy corpus such as patents. We are constantly working on improving data quality. If you find anything strange, let us know.

Can I join more metadata, such as patent assignee and title?

Obviously your first port of call would be the SureChEMBL website for patent metadata, but other services you may wish to use include the EPO web services for programmatic access.

Is there anything else?

Errr, yes. Watch this space for another post on storing and accessing live SureChEMBL data, behind your firewall. 


The SureChEMBL Team

Thursday, 12 March 2015

Beaker now officially part of ChEMBL web services


We have mentioned Beaker (a.k.a the ChEMBL cheminformatics utility web service), several times on the blog (here, here and here), but have not devoted an entire post to Beaker. Well, here it is.

Beaker - what's this?


It's a small utility, that makes chemistry software available securely over https. You no longer need to install a chemical toolkit in order to convert your molfile to SMILES or calculate descriptors. If you have an internet connection (if you can read this, chances are you do), you can use Beaker. We recommend you head over to the interactive online documentation (https://www.ebi.ac.uk/chembl/api/utils/docs), to see the full list of functionality it offers and try it with your own data.

Which toolkits are used by Beaker?

Under-the-hood Beaker is exposing the functionality of the RDKit cheminformatics library. Beaker's optical structure recognition methods use the OSRA library.

 

Do I need an API Key?


As long as you are making no more than 1 request per second, you do not need an API key. Beaker provides standard set of response headers to inform about rate limiting:



There is also one custom header:



This lets you know how you have been authenticated. The default authentication is IP-based, which means that if any other person uses Beaker from the same IP, it will affect your rate limit. This is why having your own API key can be useful - no one can 'steal' your rate limit and it will be slightly higher than default as well. If you need a key, just write to us.

I tried it and it doesn't work...


Before contacting us and submitting bug report, please try a few things first:

1. Submit data via https://www.ebi.ac.uk/chembl/api/utils/docs. It it works there, see point 2.
2. Check data encoding. Unlike the ChEMBL data web services (where you should use percent encoding as described in the previous blog post), if you are accessing Beaker via GET, then all data provided should be base64 encoded. This is why, if you want to use GET to convert 'CCC' SMILES to molfile this link won't work:  
https://www.ebi.ac.uk/chembl/api/utils/smiles2ctab/CCC.
'CCC' has to be base64 encoded first and  
base64('CCC') == 'Q0NDQw==',
so the valid link is https://www.ebi.ac.uk/chembl/api/utils/smiles2ctab/Q0NDQw==. Our online documentation will do encoding for you, and present what URL was really executed:


3. Use POST where possible. GET requests are nice, because everything gets included into URL, so you can embed such a URL in a blogpost, like we just did. One issue with GET is that there is often a maximum number of characters you can send, although this does depend on server setup. If you would like to use Beaker from ChEMBL servers for example, your link can't exceed 4000 characters. Base64 encoding will make any parameter about 1/3 longer. So for example, if you would like to send an image, in order to perform Optical Structure Recognition (OSRA), it's very hard to find a valid, good quality image, that is less than 1.2 Kb in size, so in that case using GET is not a good idea. Also, do not forget you can use curl to submit your POST requests. Below we provide some examples of how to access Beaker via POST with curl:



4. If using GET, check what type of base64 are you using. Standard implementation of base64 use the following characters:
[a-zA-Z0-9+/]
 Those two last signs ('+' and '/'), are not url-safe as they have special meaning in URLs. This is why Beaker uses url-safe version which substitutes '-' instead of '+' and '_' instead of '/' in the standard base64 alphabet. For reference, please click here.

Does ChEMBL python client library work with Beaker?

Yes, and even more it adds enough syntactic sugar to make it feel like your are using locally installed chemical toolkit. For example, look how easy it is, to compute maximum common substructure from three compounds, given as SMILES strings:


You can install the python client library by using 'pip install chembl_webresource_client' or download it here and expect more examples in a future blog post.

Does it work without the client?

Of course it does. You have already seen an example of how to use Beaker from JavaScript (using the online documentation) and python (using the client library). But because curl is very common tool, available on many platforms, you can execute calls to Beaker from your command line in bash. Bash has a very cool feature called pipes, so you can chain the output of one command to the input of another. This way you can mix calls to our data web services with Beaker calls. As an example let's assume that you have a photo of a compound. This could be a scan of the paper document, such as patent or a photo of a conference poster taken using your mobile phone, but it has to have decent resolution and quality:


Original image available here

If we would like to find the compound in ChEMBL that is most similar to the one recognized from the image above, we could use this line of bash script:



The script may look a bit hackish, but this is because we wanted to only use standard command line tools, that can be found on OSX and Linux systems. In production, we would never use grep, sed and awk to parse JSON because this is bad (instead we encourage you to try jq), but we wanted to show a nice example of using pipes to combine different tools. Anyway, the end result of running this command will be open the following page in your browser: https://www.ebi.ac.uk/chembl/api/data/image/CHEMBL2107150.


Is Beaker open source software, can I see the code?

Yes, it's hosted on GitHub (https://github.com/chembl/chembl_beaker) under a Apache 2.0 license and the latest stable version is always registered in PyPI.

This also means that you can deploy your own local Beaker version. Reasons why you might like to do this include:

1. You don't want to rely on availability of ChEMBL web services or care about rate limiting.
2. You don't want to send proprietary compounds to a public service.
3. You would prefer to install your own chemical toolkit (on only 1 machine), and access its services over http(s).

How do I cite Beaker?


Please use the following publication:




If you have any questions about Beaker or any other ChEMBL services, please let us know.

Friday, 6 March 2015

GET and SMILES interference



When reviewing our old web services documentation (https://www.ebi.ac.uk/chemblws/docs), you will notice some methods can be accessed by both: GET and POST. One thing these methods have in common is that they all accept SMILES as search parameter. Why? Well, it turns out there are some SMILES, that can not be handled via GET, when using old web services. Take this SMILES as an example: [Na+].CO[C@@H](CCC#C\C=C/CCCC(C)CCCCC=C)C(=O)[O-] and you will see, that you can't construct a valid URL using 'https://www.ebi.ac.uk/chemblws/compounds/smiles/' as a base. To 'GET' around this issue, you will need to use POST. This is a bit sad, as it means you can not put a link to such a compound on your blog or ask about it on chemistry forum or send it to your friend via Skype :( What's more, if you would like to use POST for a non-SMILES method (e.g. get all assays by ChEMBL ID), you would also be out of luck.

When reviewing the documentation of the new web services (as we asked you in the previous blog post), you will probably notice, that none of the methods mention POST support. What does it mean? First of all, it means that you can achieve everything using GET. You can easily retrieve data for the SMILES string above using GET, you just have to be careful. Placing a SMILES strings in URLs can be tricky. This is because they often contain characters that are not allowed in URLs and should be encoded, according to the URI standard. Encoded how? The standard mechanism to encode URLs is called percent-encoding and it is widely accepted by modern web browsers and other web tools, such as curl or wget.

In fact, some browsers hide the percent-encoding from users, as we will see soon. So the percent-encoded URL of our molecule looks like this:

https://www.ebi.ac.uk/chembl/api/data/molecule/%5BNa+%5D.CO%5BC@@H%5D%28CCC%23C%5CC=C/CCCC%28C%29CCCCC=C%29C%28=O%29%5BO-%5D

Not very readable... But try to open this link in Firefox and in the address bar you will see this:

https://www.ebi.ac.uk/chembl/api/data/molecule/[Na+].CO[C@@H](CCC%23C\C=C/CCCC(C)CCCCC=C)C(=O)[O-]

Much better, only one sign, #, is encoded! What if you try to open the second link in browser? In Firefox this should work. All other popular browsers will return 404 - not found and curl will complain:


So what to do?


1. These are characters, that always have to be encoded: 
  • % change to %25, because percent is a special 'escape' character in percent encoding.
  • # change to %23, because not encoded hash sign is an indicator of the Fragment identifier part of URL, which is only used in browser and is not send to server.
  • \ change to %5C, because all browsers apart from Firefox are changing not encoded backslash to forward slash as explained here.
2. This is a character, that can not be encoded:
  • Forward slash / can not be encoded as %2F because our Apache server configuration will return a 404 for URLs containing %2F. The security reasons are described in the Apache documentation. This is particularly annoying, as it means that you can not use online percent-encoders if your SMILES contain forward slashes.

 

How to make your life with SMILES simpler?

 

1. Use Firefox - this browser does a great job when it comes to encoding. Although FF is unable to guess in which context you are using % and #, so you will still encode those two characters before pasting it into the address bar, but the rest of special characters will be encoded correctly, so then you can copy the URL from Firefox to other browsers and it will just work.

2. Use cURL - just as with Firefox, you have to encode % and #, but the rest will be handled properly, you just have to use '-g' flag, which switches off the URL globbing parser:

 

How long?


You might ask yourself, will I get a valid response from the server, when I search with a really long SMILES? We can find out by first using our new web service filters to obtain the SMILES of the biggest compound in ChEMBL, the URL will be:
https://wwwdev.ebi.ac.uk/chembl/api/data/molecule?molecule_structures__isnull=False&order_by=-molecule_properties__full_mwt

And SMILES of the first molecule is:

CC.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC1OC(CO)C(O)C(O)C1O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC2OC(CO)C(O)C(O)C2O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC3OC(CO)C(O)C(O)C3O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC4OC(CO)C(O)C(O)C4O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC5OC(CO)C(O)C(O)C5O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC6OC(CO)C(O)C(O)C6O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC7OC(CO)C(O)C(O)C7O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC8OC(CO)C(O)C(O)C8O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC9OC(CO)C(O)C(O)C9O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%10OC(CO)C(O)C(O)C%10O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%11OC(CO)C(O)C(O)C%11O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%12OC(CO)C(O)C(O)C%12O)C(O)CO.CCCCCCCCCC(C(=O)NCCc%13ccc(OP(=S)(Oc%14ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%14)N(C)\N=C\c%15ccc(Op%16(Oc%17ccc(\C=N\N(C)P(=S)(Oc%18ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%18)Oc%19ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%19)cc%17)np(Oc%20ccc(\C=N\N(C)P(=S)(Oc%21ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%21)Oc%22ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%22)cc%20)(Oc%23ccc(\C=N\N(C)P(=S)(Oc%24ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%24)Oc%25ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%25)cc%23)np(Oc%26ccc(\C=N\N(C)P(=S)(Oc%27ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%27)Oc%28ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%28)cc%26)(Oc%29ccc(\C=N\N(C)P(=S)(Oc%30ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%30)Oc%31ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%31)cc%29)n%16)cc%15)cc%13)P(=O)(O)[O-]

This is huge! But, more importantly, the SMILES looks like it's already url-encoded, as it contains %23 and %29, which will be interpreted as reserved characters and translated to # and ), when  pasted into a browser address bar. To prevent this from happening, we must first percent-encode the percent sign, to ensure they are interpreted literally. Percent is encoded as %25 and after conversion our URL will look like this (this URL contain many backslashes so it will only work in Firefox):

https://www.ebi.ac.uk/chembl/api/data/molecule/CC.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC1OC(CO)C(O)C(O)C1O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC2OC(CO)C(O)C(O)C2O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC3OC(CO)C(O)C(O)C3O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC4OC(CO)C(O)C(O)C4O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC5OC(CO)C(O)C(O)C5O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC6OC(CO)C(O)C(O)C6O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC7OC(CO)C(O)C(O)C7O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC8OC(CO)C(O)C(O)C8O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC9OC(CO)C(O)C(O)C9O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%2510OC(CO)C(O)C(O)C%2510O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%2511OC(CO)C(O)C(O)C%2511O)C(O)CO.CCCCCCCCCCCCCCCC[NH2+]OC(CO)C(O)C(OC%2512OC(CO)C(O)C(O)C%2512O)C(O)CO.CCCCCCCCCC(C(=O)NCCc%2513ccc(OP(=S)(Oc%2514ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2514)N(C)\N=C\c%2515ccc(Op%2516(Oc%2517ccc(\C=N\N(C)P(=S)(Oc%2518ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2518)Oc%2519ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2519)cc%2517)np(Oc%2520ccc(\C=N\N(C)P(=S)(Oc%2521ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2521)Oc%2522ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2522)cc%2520)(Oc%2523ccc(\C=N\N(C)P(=S)(Oc%2524ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2524)Oc%2525ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2525)cc%2523)np(Oc%2526ccc(\C=N\N(C)P(=S)(Oc%2527ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2527)Oc%2528ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2528)cc%2526)(Oc%2529ccc(\C=N\N(C)P(=S)(Oc%2530ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2530)Oc%2531ccc(CCNC(=O)C(CCCCCCCCC)P(=O)(O)[O-])cc%2531)cc%2529)n%2516)cc%2515)cc%2513)P(=O)(O)[O-]

It should be noted, that the URL is 1666 characters long, which is below the maximum allowed URL length in our Apache setup (4000 if your were wondering). Even if we encode all the special characters, we will end up with the following URL (this time valid in all modern browsers):

https://www.ebi.ac.uk/chembl/api/data/molecule/CC.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC1OC%28CO%29C%28O%29C%28O%29C1O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC2OC%28CO%29C%28O%29C%28O%29C2O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC3OC%28CO%29C%28O%29C%28O%29C3O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC4OC%28CO%29C%28O%29C%28O%29C4O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC5OC%28CO%29C%28O%29C%28O%29C5O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC6OC%28CO%29C%28O%29C%28O%29C6O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC7OC%28CO%29C%28O%29C%28O%29C7O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC8OC%28CO%29C%28O%29C%28O%29C8O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC9OC%28CO%29C%28O%29C%28O%29C9O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC%2510OC%28CO%29C%28O%29C%28O%29C%2510O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC%2511OC%28CO%29C%28O%29C%28O%29C%2511O%29C%28O%29CO.CCCCCCCCCCCCCCCC%5BNH2+%5DOC%28CO%29C%28O%29C%28OC%2512OC%28CO%29C%28O%29C%28O%29C%2512O%29C%28O%29CO.CCCCCCCCCC%28C%28=O%29NCCc%2513ccc%28OP%28=S%29%28Oc%2514ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2514%29N%28C%29%5CN=C%5Cc%2515ccc%28Op%2516%28Oc%2517ccc%28%5CC=N%5CN%28C%29P%28=S%29%28Oc%2518ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2518%29Oc%2519ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2519%29cc%2517%29np%28Oc%2520ccc%28%5CC=N%5CN%28C%29P%28=S%29%28Oc%2521ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2521%29Oc%2522ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2522%29cc%2520%29%28Oc%2523ccc%28%5CC=N%5CN%28C%29P%28=S%29%28Oc%2524ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2524%29Oc%2525ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2525%29cc%2523%29np%28Oc%2526ccc%28%5CC=N%5CN%28C%29P%28=S%29%28Oc%2527ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2527%29Oc%2528ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2528%29cc%2526%29%28Oc%2529ccc%28%5CC=N%5CN%28C%29P%28=S%29%28Oc%2530ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2530%29Oc%2531ccc%28CCNC%28=O%29C%28CCCCCCCCC%29P%28=O%29%28O%29%5BO-%5D%29cc%2531%29cc%2529%29n%2516%29cc%2515%29cc%2513%29P%28=O%29%28O%29%5BO-%5D

which is 2478 characters long and still lower than 4000 limit. This means we can use GET to retrieve every ChEMBL compound based on its SMILES string.

 

What about POST?


OK, so SMILES can be passed via GET, can POST still be used? Yes! For every single method you can use POST as well. This is important, as it is possible to create very long URLs, by chaining together multiple filters and other parameters, such us pagination and ordering. This is why all new web services methods (even those, which don't expect any SMILES), can be executed using POST. In fact, our updated Python client (blog post coming soon!), is using POST for almost everything.  Just keep in mind, that in order to use POST to retrieve data, you have to add X-HTTP-Method-Override:GET header.

If the earlier discussion around SMILES encoding all seems a bit too complicated (hence the xkcd comic strip) or just too much hassle, you can always use POST. But remember you won't be able to share SMILES links in your blog posts or Skype conversations :(

 

Is using POST to retrieve data 'breaking the web'?


There was a recent discussion about this topic after Dropbox team announced (just like us), that they see some limitations in GET compared to POST and will start to allow POST methods to retrieve data (original article). There were some critical opinions about allowing POST access to data, stating that this is a poor API design. This is turn has triggered a long discussion on hacker news page, from which one comment is particularity important:
  
"I really like how the Google Translate API handles this issue.  The actual HTTP method can be POST, but the intended HTTP method must always be GET (using the "X-HTTP-Method-Override" header)." 

And this is exactly what we are doing, and we also believe this is the right way to use POST in order to allow retrieval of data from RESTful web interface. 


The ChEMBL Team