API responses suddenly missing emailAddress field

[Roel Abspoel]

I’m using the ‘Get users in organization’ method in the servicedesk API as documented here. (although the same goes for any other method that should, according to the docs, return e-mail addresses)

The example result shows a field emailAddress, however this field is missing in the response I get from this method.

As suggested elsewhere I checked the global setting User email visibility which is set to Logged in users only but the results are the same when set to public.

The user I’m connecting with also has the Browse users and groups permission.

Somehow the emailAddress field went missing in all API calls in the last few days suggesting it is because of some setting that was changed but I can’t figure out what setting is responsible for this.

Can anyone give me insight in why this field is (suddenly) missing from my API results? (and on how to get it back since its vital to our synchronization logic)

TL;DR answer

The missing field(s) are by design; it's not a bug but a choice made to comply with GDPR. Individual users can choose to disclose their email through the API but this means some will and some wont. A new API will be added in the future where you can get a list of all e-mails, however this requires whitelisting of your application. If this change breaks your code, then the only solution is to change the logic OR use the new API (when its available) OR have all your users disclose their email. For more information, see the links in the official answer below.

[Dario B · Accepted solution]

Hello,

This change is related to:

• Upcoming changes to Atlassian account profiles

More details about this change have been shared in the developers community:

• Guidelines for Testing Profile Visibility Controls
• Guidelines for requesting access to email address

Please review above 2 threads in community.developer.atlassian.com and add your questions to there, if any.

Cheers,
Dario

[Navarambh Software - Gmail]

Is the "new profile visibility controls" section available yet? We are not able to find that screen anywhere.

So at this point, is there no way to get the email address unless the email API access request is approved ? 

[Dario B]

Hi @Navarambh Software - Gmail ,

I can see that this question has already been answered in the support ticket you have created for this issue:

[...] we’re gonna add (in a few days) a flip switch for each user to choose the visibility of their email address. [...]

Also, if this is very urgent for you and you need to have a workaround applied, I have been told that there should be a way to temporary disable this, even if it is not advisable. Please discuss this option with the support engineer handling your ticket since if you really cannot wait. 

I hope this helps.

Cheers,
Dario

[Navarambh Software - Gmail]

Thanks. Really wish that the new controls were already in place before removing the email address. There must be a good reason to hastily remove the email address without providing the associated screens with it.

[Dario B]

You may want to provide this feedback to the developers community threads announcing this change.

[Ed LeBlanc]

Having exactly the same issue with missing 'emailAddress' value as others in this thread.  

This has become an outage situation (yet another from jira)

The links provided by Dario Bonotto do not help, and only add more questions. 

[Roel Abspoel]

In short: the missing field(s) are per design to comply with GDPR. So the fields are gone and will remain gone. Users will be able to allow you to use the fields, however this is per user so might not be useful for everyone. (if you synch users then you need ALL emailAddresses, not just some)

There will be a new API that allows you to get the addresses, however you will have to get white listed for it. (Your app needs to comply to GDPR basically)

In all other cases you will need to find a new way to do what you want to do. We are now experimenting with account Id's instead which is a pain but for now should work. But miles may vary depending on needs.

I think that even though it was communicated it was done in a poor fashion; at least put this information in the API documentation since that is the goto source for all developers. But it is what it is I guess ;)

[Dario B]

Thanks for taking the time to write down this answer @Roel Abspoel !

Also, I will share your feedback with the team taking care of the REST API documentation. 

Have a nice day

[Ed LeBlanc]

Yes, thank  you @Roel Abspoel for the concise explanation.  

"the fields are gone and will remain gone"

"need to comply to GDPR for whitelist api"

"communicated in a poor fashion".  

This gives a direction for how to fix - re-code.  The emailAddress was the only link we had between jira and our local system for matching up users, so now we need to figure out something else - during the outage this caused.  

I'm still waiting for Jira support to get back with me, and hopefully confirm this info.  

[Dario B]

@Ed LeBlanc 

I see you have a ticket open with Jira Support on this but, as written on the bottom of the Guidelines for Testing Profile Visibility Controls thread, it would be best to open your  request with the Developers Support team instead:

Raising issues with Atlassian

• Report bugs here.
• Ask questions about GDPR on the Developer Community or with our Developer Support team.

I hope this helps.

Dario

[Dario B]

Also, @Roel Abspoel @Ed LeBlanc @Navarambh Software - Gmail ,

Please notice that this change has been actually announced in different places and emails have been sent to all the admins:

• Major changes to Jira Cloud REST APIs are coming to improve user privacy (May 2018)
• Major changes to Atlassian Connect APIs are coming to improve user privacy (Jun 2018)
• State of Atlassian Ecosystem changes to address privacy (February 2019)
• Upcoming changes to Atlassian account profiles (May 2019)

Also, as advised, we are now adding all the relevant documentation to the REST API pages. This is now mentioned on the top of the already provided threads in the developers community:

Note: These guidelines are still being drafted and when finalized will be published in our developer documentation. This information is subject to change. We wanted to get this information to you as soon as possible, even in draft state, to help better prepare you for testing.

Finally, be also aware of the below bugs when handling non-managed users that have just been reported:

• JRACLOUD-72483 - REST API endpoints that return user objects leave email field blank for unmanaged users
• JSDCLOUD-8289 - API: Unable to fetch customer email address via API
• ID-7019 - Setting Account visibility to "Public" or "Collaborators" does not allow email address to show in API or UI

Cheers,
Dario

[Navarambh Software - Gmail]

@Dario B Is the new Privacy screen available yet?

[Dario B]

@Navarambh Software - Gmail  

Apologies for late reply, I am just back from vacation.

However, the Profile Visibility Settings screen is already available since couple of days after this thread was started.

Can you kindly confirm you can see it? 

Also, since this can be useful to you and other users as well, please find below summarized the expected behavior for each setting:

----------------------------------------------

• anything set to public will always be visible regardless of the type or caller or the authentication method
• anything set to the user's organisation will be visible to:
• users in the UI
• users doing rest calls authenticating as themselves
• it won't be visible to connect apps*
• it won't be visible in webhooks (since they are not authenticated)
• anything set to "only me and admins" will be visible to:
• the user when they request themselves (apart from connect apps*)
• in the user management UI (admin.atlassian.com) it will be visible to admins in the user list (but nowhere else)
• if its a managed account it might also be accessible to admins through the identity user managment api (not to be confused with the Jira REST API, this is a different  one): https://developer.atlassian.com/cloud/admin/user-management/rest/#auth 
• customer accounts email will be visibile:
• to agents in the UI on profile cards
• to agents in places using user search APIs inside Jira (there is no clearcut rule for which places it is), including the user search REST API, but not the user lookup API
• it should be visible to all agents everywhere (apart from connect apps*)

* connect apps can apply through ecosystem to use some special api endpoints to get email regardless of privacy settings. This has to be approved by legal and is only for apps in specific use cases

----------------------------------------------

Cheers,
Dario

[Josh Bazemore]

We're having the same problem for an internal use-case. Is there a timeline on when the new profile controls will be rolled out?

[Navarambh Software - Gmail]

@Roel Abspoel @BlueAnt 

Are you getting any response from Atlassian support on this issue? We are having a hard time getting any meaningful response.

[Navarambh Software - Gmail]

@George Skryl

Do you have any update from Atlassian on the issue?

[George Skryl]

@Navarambh Software - Gmail

No, we're working on a workaround instead.

[Navarambh Software - Gmail]

@George Skryl Can you please provide us some details of the workaround? It will be hugely helpful to us.

[George Skryl]

@Navarambh Software - Gmail It really depends on your case. We were using emailAddress to link user accounts between JIRA and our system. After some research we decided to replace it by Atlassian Account ID. You can find more here https://developer.atlassian.com/cloud/jira/platform/deprecation-notice-user-privacy-api-migration-guide/. I hope this would help.

[BlueAnt]

I have the same problem, synchronization with other system is not working anymore due to the issue.

[Navarambh Software - Gmail]

We are facing the same issue. Causing an outage for us in production.