This blog was originally written by Doug Lerner, of Elliptics.com
Introduction
We implemented an API which allows Eldis Communities members to link their LinkedIn profiles with their Eldis Communities profiles. That user experience and implementation is described in detail in the document “LinkedIn plug-in”
In May 2015 we suddenly discovered that LinkedIn began blocking our API requests for profile data. It emerged that this was because they had decided to restrict usage of their API to those meeting a new set of criteria, apparently to curb the commercial value third parties could generate. This document discusses what happened, what those new restrictions were, how we coped with the sudden change and the risks involved with depending on third-party APIs for content.
Profile data available from LinkedIn
LinkedIn makes various APIs available which allow developers a chance to offer their members an option to allow access to parts of their profile information (fields) for certain restricted uses.
Current LinkedIn policy makes the following fields available through these APIs to all developers (ref: https://developer.linkedin.com/docs/fields ):
Basic Profile Fields, Location Fields, Position Fields
Eldis Communities makes use of the following fields from within the Basic Profile Fields, Location Fields and Position Fields options.
| LinkedIn Field | How used at Eldis Communities |
| First-name | Stored only |
| Last-name | Stored only |
| Headline | Shown in profile, replacing Job Title |
| Location | Country code is stored only; a descriptive location and country name are shown in profile, replacing Country of Residence |
| Industry | Stored only |
| Num-connections | Stored only |
| Summary | Shown in profile, replacing About Me |
| Positions | Shown in profile, replacing Position Type |
| Public-profile-url | Shown in profile, points back to the member’s LinkedIn profile |
However, prior to the sudden changes in API availability, which occurred just at the time of our public release, the following fields were also retrieved: interests, publications, languages, skills, educations, and num-recommenders.
Those fields are now available only to applications which have been approved by the Apply with LinkedIn program and include:
Full Profile Fields, Contact Info Fields, Company Fields, Publication Fields, Patent Fields, Language Fields, Skill Fields, Certification Fields, Course Fields, Education Fields, Volunteer Fields, Recommendation Fields
History of the LinkedIn changes during Eldis Communities development
The changes to the rules came with some notice, but nonetheless occurred in a way which caught us by surprise, because we felt we were in full compliance, both in spirit and with the letter of the rules.
Here is a summary of some of the key milestones in our development and how they coincided with notices from LinkedIn about changes to their API rules, and the impacts on the system we created.
15 December 2014 – Our own research began on the use of LinkedIn APIs.
25 December 2014 – Initial LinkedIn development took place involving API application registrations and the generation of API keys with LinkedIn.
31 December 2014 – Development work continued on OAuth 2.0 authentication, including initial successful tests of the authentication process, and we added code to handle the canceling of a request, and the case of users not granting permission
9 January 2015 – Creation of new PHP server to handle communications between Eldis Communities and LinkedIn and initial success with API calls to LinkedIn.
10 January 2015 – Our LinkedIn API v1 was operational.
26 January 2015 – We updated to our LinkedIn system including the addition of new essential fields which were requested, converting “collections” (structured LinkedIn data) to JSON representation for passing back to Eldis and successfully generating reports based on data stored in the Eldis Communities database.
28 January 2015 – At this point we had a working system and had added more requested fields, including publications details for members who had publications in their LinkedIn profiles.
3 February 2015 – We began work on a new “light box” UI to enhance the user experience of using our LinkedIn API, including light boxes to grant permission, request reminders, show confirmations and otherwise make the overall experience more seamless during the redirection process between Eldis Communities, the intermediate API server and the LinkedIn site.
10 February 2015 – Completed work on ironing out various issues with the flow through light boxes and added the ability to return to any location and not just to the user’s personal space.
10 February 2015 – Finished a customized profile view to allow the integration of LinkedIn data in a user’s profile view.
12 February 2015 – We received an email from LinkedIn informing us that changes would be made to the Developer Program that would like affect our LinkedIn API access (see screenshot).
13 February 2015 – After examination of the rules, we had an internal review of the LinkedIn API terms of use (https://developer.linkedin.com/legal/api-terms-of-use) and concluded we remained in compliance because of the clause shown in the following screenshot:
This was precisely the definition of our use case, so we believed things were okay going forward.
25 March 2015 – Work began on integrating the LinkedIn data collected with the other GOKH components, mainly in providing the data to the SOLR index.
8 April 2015 – We completed integration of LinkedIn data in special <linkedin> tags for export to the SOLR index.
18 April 2015 – Customized profile edit working for members who have integrated LinkedIn profiles.
20 April 2015 – Added a reporting feature added to show the status of eligible members who have completed, denied or have been reminded about the LinkedIn feature.
24 April 2015 – Added a new section of profile edit options visible to LinkedIn users which allowed such users the ability to edit their LinkedIn profile, refresh their LinkedIn connection and also revert to using just their Eldis Communities profile.
4 May 2015 – Added a LinkedIn profile refresh link on user profile pages.
13 May 2015 – We released the current LinkedIn features to all eligible community members, including new profile edit, options to revert and switch, etc.
13 May 2015 – Also on this same day we received another email from LinkedIn about the API access changes, as shown in the following screenshot:
13 May 2015 – The LinkedIn API suddenly ceased to work. We immediately contacted LinkedIn regarding this, as shown in the following screenshot.
The text of our inquiry, which is difficult to read in the screenshot, was the following:
Application Information:
What company will be using this application (if different from the development company)?: Eldis Communities (IDS – Institute of Development Studies)
Please provide a general description of what this company does, industry type, etc.:
Eldis Communities, part of the Institute of Development Studies, at the University of Sussex in the UK, is an NGO sponsored non-profit which provides a social network for researchers and workers in developing countries.
I checked all the conditions of the new policy and we are 100% in compliance which each item.
All we are doing is letting members supplement their regular Eldis Profile with LinkedIn profile info they grant permission.
The API suddenly stopped working today and it’s a disaster. We beg you to please review this as soon as possible! We’ve been developing this for months and just on the day we are releasing new features it stopped working because we are using r_fullprofile. Please help!
What Consumer/API key are you requesting this permission for?: 75j9qlo4v8rkdp
How can we access your application to review it for program compliance?:
You can sign up for an account at [link removed]. So far we have only released the LinkedIn API for one group at Eldis. After creating an account, please email me at [link removed] and I will give you access to the group and you can see exactly how it works.
If needed you can also call me at 617-379-0363.
What is the status of your application?:
(x) Production
( ) Development
( ) Other
13 May 2015 – The following reply was received in response to our request for assistance:
Due to changes in the API program we no longer offer Apply with LinkedIn outside of specific instances. It is only permissible for companies to use it to allow members to apply for jobs at that specific company. It can not be used in any other type of application. You can review the changes here: https://developer.linkedin.com/docs/apply-with-linkedin.
You can apply for the Consumer Partner Program and get access to the functionality you need there. Here’s the link: https://developer.linkedin.com/partner-programs/consumer
Best of luck,
Apply with LinkedIn Review Team
14 May 2015 – We then made an application for the Partner Program in an attempt to regain access to the full functionality, as shown in the following screenshot:
21 May 2015 – We received the following reply informing us that our application for the Developer Program had been rejected.
21 May 2015 – We again contacted LinkedIn and attempted to convince them that we fit all the eligibility criteria:
21 May 2015 – Adrian also attempted to contact LinkedIn to further explain why we believed we matched the LinkedIn criteria perfectly and attempt to regain access to the API.
22 May 2015 – LinkedIn again replied and rejected our request:
22 May 2015 – Adrian once again attempted to find out more about why we did not qualify and why our application was rejected:
26 May 2015 – LinkedIn once again replied, but just to say they felt we did not qualify, with no further explanations, and to suggest we continue just using the more open API we had integrated via the open program.
11 June 2015 – We once again attempted to elicit some detail about where our use case failed to qualify for enhanced API access.
18 June 2015 – After no longer receiving replied from LinkedIn we decided to proceed with the more limited open API and informed Peter of such:
25 June 2015 – Rebirth of the LinkedIn PI to make use of the new restricted scope, now limited to just r_basicprofile, according to the new LinkedIn rules.
13 July 2015 – SOLAR LinkedIn tags updated.
29 July 2015 – Implemented a new LinkedIn eligibility test to expand the number of Eldis Communities members who have access to the API while controlling against members who should not (members only of excluded groups).
30 July 2015 – Basically the existing LinkedIn connection was made live except for minor tweaks going forward, creation of additional reports, and other adjustments required as other new features (such as spammer blocking) were added to Eldis Communities.
Risks of using third-party APIs
In retrospect, it turns out our experience with LinkedIn was nothing new to LinkedIn and also happens other well known companies which provide APIs. It seems to be a point of continued anguish for developers wishing to provide features from and integrate with other platforms.
Some examples of this include:
LinkedIn itself. In July 2013 LinkedIn made changes to its API to limit the time a third-party app could access a user’s profile to 60 days, with no option for the user to choose to extend the access. Developer Roger Lee, who created the once popular “Job Change Notifier” – a LinkedIn app which had 100,000 users – was forced to shut down the app. This is considered an instructive example of users suffering when later-stage companies begin to restrict access to their APIs in order to drive their monetization strategies.
LinkedIn again. In April 2014 changes made to the LinkedIn API caused consternation amongst other API developers. ZDNet reported (http://www.zdnet.com/article/linkedout-crm-companies-squawk-over-linkedins-api-policies/) that Zoho and others ended integration with LinkedIn after the changes because the new developer partners turned out to be Microsoft and Salesforce.
Facebook. In April 2015 Facebook made changes to APIs which crippled connections Microsoft built into its mobile and desktop apps to link to Faceboook calendars, contact and photos. (http://www.computerworld.com/article/2935627/social-media/facebooks-api-change-severs-microsoft-apps-from-friends-info.html)
Google. The Google Translate API, which was also implemented at Eldis Communities, was scheduled to be shut down in March, 2015. However, at the moment it continues to work. A similar shut-down of an earlier version of the Translate API also occurred in 2011. Google said it was because of “extensive abuse” by users of the service. (http://www.itproportal.com/2011/05/27/google-close-translation-api-service/)
Obviously there are risks when dependent on third-party APIs. One take-away from this is that use of such APIs should be considered adjuncts to existing services, and not become key services themselves. Since we have no control over them, we should not build “existential” functionality around them.