JIT provisioning

Lets take a break from the check lists and take a look at another interesting subject: Just In Time (JIT) provisioning.

Over the last couple of years SAML has emerged as the defacto standard for federated authentication and authorization. If you are working with a partner the first question is usually "Do you support SAML?".

Incoming SAML makes it possible to essentially outsource the process of authentication and authorization to a business partner. The partner vouches for the identity of the user and you can essentially use this information to give the user access to your system. This solves the run time but in most cases you still need a "back channel" provisioning process. Getting a SAML assertion telling you that the user "msandr01" would like to log in to your application  logging is good but most application needs more information to create a working system account.

Nishant Kaushik published a very good four part series of blog postings on this subject about a year ago that I highly recommend. I ran into the problem in a discussion at Wisegate a couple of weeks ago and I am also looking into the problem for a couple of use cases at work.

The use case we talked about at Wisegate was a bank that had outsourced all of their customer facing applications to various vendors. One vendor did retail banking, another brokerage, a third investment banking etc. All of the different vendors had their own user repositories and own SSO solutions so a single bank customer could have multiple logins and multiple passwords and would have to login to each application separately. The business of course didn't like this and wanted an SSO solution.

The true high tech JIT solution would be to use a federated authentication product such as IBM TFIM and do SAML with the apps all in run time. The hub would be truely light weight and not persist any information about the users. A typical user interaction with the hub would look like:

1. Take the request from the user
2. Figure out which application this user is trying to login to
3. Figure out if the user has any account in any of the apps by asking the apps
4. Authenticate the user
5. Authorize the user
6. Create the SAML assertion and send it to the app
7. Act as a reverse proxy in the interaction between the user and the app

In theory this is a great idea but there are some practical considerations.

One issue is latency. Given that this is an online person facing transaction the login should ideally not take more than three seconds (or so) and if we end up pushing 10-15 seconds the business will start screaming. The SSO hub and the apps are physically in different places which means that you will get latency even if you have lighting fast machines that process the requests.

Another issue is complexity. This does require quite a lot of bleeding edge technology and there are plenty of things that could go wrong.

In the end the discussion ended with the conclusion that it is probably safer to go for a more conventional approach where user populations of the apps are reconned back to a central repository in the SSO hub using a meta directory product. SAML would still be used to communicate assertions to the applications but this solution is a lot faster and eliminates a lot of the unknowns. This solution pattern is very common among a number of vendors including Symplified.

Not as high tech and cool but is guaranteed to work and won't cause hard to fix latency based performance problems .

Checklist manifesto part two - requirements gathering

This posting is a continuation to Checklist manifesto. In that post I discussed how the concept of checklists can be applied to IAM projects on the overall delivery methodology level. Lets talk a bit about how check lists can be used in the different parts of the delivery methodology.

Lets assume that you are using a classical waterfall. This gives you the following steps:

1. Requirements gathering
2. Design
3. Implementation
4. Test
5. Go live
6. Maintenance

In this post I will focus on how you would user checklists in the requirements gathering phase.

One thing I have noticed over my IAM implementations is that if you take a use case driven approach it seems like most provisioning projects will contain almost the same use cases. Depending on how you slice and dice your cases and what your scope is you usually end up with 30-50 core use cases which tend to cover the same subjects.

The use cases may be very different as each and every company seem to like to do things their own special way but you will cover the same overall business process.

This means that a mature implementation organization should be able to come up with a list of use cases that can be given to the more junior resources that will perform the actual requirements gathering. If you are a customer I would definitely include this as a question on the RFP. If you are a junior resource I would speak to your seniors and check if they don't have a list of use cases on their hard drives or if they quickly can create one based on their previous projects.

More about my experiences in the lovely world of requirements gathering can be found in the post UAT and requirements gathering.

Checklist manifesto

Management books are often boring and mostly not really applicable on your situation so they rarely makes good material for a blog post. A while ago my wife was told to read "The checklist manifesto" by her boss and once she was done with it I took a look and actually really liked it.

The core concept in the checklist manifesto is that there are certain series of rather simple steps that needs to happen in order to perform a complex process. The core example is the surgery checklist but Atul Gawande (the author) also uses examples from other fields such as aviation.

Can this principle be used in identity and access management projects? I would say definitely yes.

Good examples why checklists are useful are my two first IAM projects. In the first "first project" I basically had no clue on what I was doing neither on the process side nor on the technical side. Luckily I was in a very junior position so my lack of knowledge didn't doom the project. Interestingly many of the senior resources also didn't have any knowledge of the product but luckily the client realized this and hired a very seasoned person straight from the product manufacturer who managed to get the project back on track and utilized the quite impressive domain knowledge of the senior resources to create a very good solution. The project also managed to pick up some very talented technical resources along the way which helped quite a lot.

In my second project I had a very senior role. In fact I really wasn't ready to take on the role and the project suffered from my lack of experience. We as an organization also had some other issues that we had to work out and in the end I as well as the organization ended up being much stronger but it took a lot of hard work.

In both projects checklists we ended up creating checklists. In the first the main contractor already had a semi formal checklist for how to run IAM projects. They didn't know how to run an OIM project but they applied their general IAM checklist to the project and where decently successful. The initial design that they created was totally unimplementable in OIM but at least they had a design that after a few months of tweaks and changes ended up being implementable.

In the second project we didn't really have any checklists or any form of process until we got some help from one of the senior PMs. He had a general checklist for how to do IAM engagement and we also developed a form of general checklist for how to do offshore development engagements in IAM projects.

Once we had our checklists in place for how to develop, test and migrate the code we could start delivering code that did what the design said it would do. Now the problem morphed to gather the correct requirements so that you can create a design that solves the actual business problem. The requirements and design gathering process is much harder to codify so that was much more of a challenge and it took me a few more years to get to a point where I think I am getting a good handle on that challenge.

Pass through authentication

One of my readers remarked that one the hardest technical challenges is to migrate things that can't be migrated.

One prime suspect here are password hashes. In most well designed systems you don't store passwords in encrypted in a reversible format but rather in form of one way hashes (preferably with some salt mixed in). This means that the only way to migrate the passwords is through mass cracking which usually isn't feasible or at least shouldn't be feasible.

In TDS there is a very interesting solution to this problem in form of pass through authentication. You essentially let the password field be empty and specify that when the user tries to authenticate simply authenticate against the old system. If the authentication is successful then set the new password in TDS. Very good solution and the design patterns can be easily implemented even if your "authentication repository of choice" doesn't support this functionality natively.

Sun set

One of the things my professor in system development at Chalmers told me was to never put anything related to "new" or "next generation" in a system name because at some point the system will be old and will need to be replaced. Having to name the next system "even newer" simply looks silly.

Currently there are a lot of Sun IDM owners that are contemplating upgrading into something that actually is moving forward. Oracle has promised continuing support and probably will deliver, at a steep price, but with the platform not being improved investing more money to add new capabilities doesn't make sense. It will also be harder and harder to find staff that can support the system. I recently heard about a Sun client where the staff had spent three months trying to install a new instance of the access manager agent. The client simply couldn't get it to work and had to give up in the end which of course resulted in some serious angst among the senior leadership as it meant that if the access management agent broke in production the entire application would be out of order for at least an extended period of time and potentially permanently.

We are currently in the final part of the first phase of our migration of Sun and I thought it may be interesting to talk a bit about the experience.

When you do a major upgrade of an IDM stack you basically have two choices. The first approach is to try to do some form of automated or semi automated upgrade. You essentially look at the upgrade as a big patch so you apply the patch in dev and do some regression testing. If everything looks good you start to walk the environments through test, stage and finally prod. In theory this works well and there are even cases when it works in practice (see MIIS to ILM). In the case of going from Sun IDM to Oracle IDM I think it would take a small miracle and a very skilled team to get this approach to work. The product stacks are simply too different and too complex so there is going to be cases where the conversion simply isn't possible. If you also have substantial customizations in play I would consider the approach being close to setting yourself up to failure.

The second approach is to try to leverage as much of your already existing documentation but essentially look at the project as a refactoring exercise. If you have good business requirements and workflow documentation then use that but anything from design and upwards I would strongly recommend that you rework. Most IDM installations don't have good requirements to start with and given that you usually end up creating delta releases rather than updating the original documentation even the ones where the docs where good at one point usually are a victim of "here is the original docs and here are the docs for the 5-30 delta releases that we have put in over the last five years" syndrome.

Update: Identigral's blog recently covered this issue. It is interesting to see that their conclusions are quite closely aligned with this posting.

XACML for eHealth

If you have been reading this blog for a while you may have noticed that I have an interest in XACML and especially in using XACML in the healthcare sector.

If you have similar interests you may be want to take a look at a webinar that Axiomatics will be running on May 5 2011.

CISSP

There are certain things in life that you plan to do for a very long time before you actually get around doing it. In my case CISSP is one of these things that I have thought about doing for more than five years but something has always interfered. In December I finally managed to attend a test and it went well so now I am a CISSP.

I actually didn't study very much for the test as the twins eats most of my spare time. I did take the time to read through Shon Harris All In One Exam Guide which I think helped quite a bit. Really good reading and actually worth taking a look at even if you are not going up for a CISSP.

The best piece of advice I got was to remember to bring some snacks and something to drink. The exam is six hours and you really need something to keep you from keeling over of exhaustion. It was many years since I have ever felt as totally knackered as I felt after the test.

Monitoring

Most enterprise IDM systems are very complicated and complex beasts with redundancy in not only the presentation layer but in the application layer and the data persistence layer. This can make it hard to answer the simple question "Is the system working properly or not". In most cases you also want to be able to spot issues early so that you can fix them before they become a problem that may take down the entire system.

The best way to ensure that all components are healthy and all services are up is to implement a comprehensive monitoring program so what are the things that you tend to want monitor?

The most basic monitoring policy is the "wait until the end user yells" approach. In this approach you simply wait for the end user to start screaming and as long no one is screaming then things must be fine. This approach have some significant limitations so it is not the way I would recommend.

Once you start talking monitoring you usually discover that the corporation has some kind of standardized monitoring tool that you should/must use. These tools usually can provide the following functionality:

• Host monitoring (is the server OS up)
• CPU/Memory/disk monitoring
• Process monitoring

Monitoring can be done either with a monitoring agent that is installed on each server or through an agentless approach. Having basic infrastructure monitoring can be very useful as it will alert you about creeping issues such as small memory leaks or logs that slowly but steadily eats up all available disk space. The trick is to make sure you actually can fine tune the alarm threshold and response level as you go along. In most cases you do want to be told if the CPU suddenly spikes from a max of 25% to 95% but as being woken up at 2 am every second Wednesday because the CPU load spikes for a few seconds during a batch load may not be ideal for you (or your marriage) you do want the ability to put in exceptions in the logic.

In most cases you will have some kind of network or port monitoring as part of your load balancer setup. Given that port oriented network monitoring configuration tends to get very complex I will write about this in a separate post.

Next step is to look at the application aware monitoring. This is usually accomplished by looking at the application logs and escalating entries that are following a certain pattern (i.e. whose log level is ERROR). You can also look for specific error messages that you know are thrown when a specific error condition occurs.

Once you have monitoring in place you should be able to sleep better at night. At least as long as your monitoring doesn't wake you up reporting nuisance errors.

UAT and requirements gathering

In provisioning projects requirements gathering and UAT testing is always an interesting area. The general problem with UAT and system level testing is that in order for the testing to be useful you really need to know what result is the correct result and what is a failure.

When you write bespoke software you can usually find the answer to this question in the design which in turn is derived from the functional requirements which are derived from the business requirements. In a typical IAM implementation on the other hand you tend to use very feature rich platforms that provides huge amounts of functionality most of which you will never use in any given project. It is also often hard to shut down this "bonus functionality" so the functionality often is available even if it really shouldn't be used. Many times when you get to UAT the testers starts touching all kinds of buttons and levers and unless you have an experienced team that has spent substantial amount of time on actively locking things down you will have some issues with this.

Business process wise IAM implementations can fall on anywhere scale ranging from totally transformative to literal refactoring where the business process doesn't change at all. In the first case the challenge is that the UAT needs to reflect the new business process and you also need to ensure that the new business process actually supports all the functions that the business needs. In the second case things are easier because you basically just need to capture an already existing process.

No matter the strategic focus of the IAM project a proper UAT needs to be business process focused. The whole point of UAT is to ensure that all business critical processes are present and are working as expected or at least in a way that is acceptable. The irony is of course that if you discover substantial process breaks in the UAT you are usually in deep trouble as it usually is too late to fix things before the planned go live.

The best way to avoid this situation is to ensure that you do a form of dry run UAT very early in the process. In my experience one of the best ways to do this is to utilize simple Visio workflows. You start with the requirements, which usually are more or less useless, and create Visio workflows. You then show the workflows for the people that knows or should know the business process. These people can rarely tell you what is needed but they can usually tell you if you get it wrong or tell you things that you should take into considerations. After a couple of cycles you usually have a pretty good process and you have also gained the buy in from one of the stakeholders.

If your architect is experienced he or she can usually produce a pretty good set of flows that can act as a starting point. Most more sophisticated consultancy organizations will be able to provide a standard set of flows as well. If your implementation team starts coding without first establishing, vetting and communicating the business process it is time to start getting alarmed. The project may still deliver successfully but it will most likely have a very painful UAT or even worse a very painful go live in front of it.

Pulse 2011

I will be in a panel at Pulse 2011 so if you want to hear me and some very distinguished copanelists talk about IAM you can visit session 1925 Identity and Access Management 2-3 pm on Mon Feb 28 in room 123 MGM Conference Center level 1.

My talk will be about the IAM challenges that BCBS MA currently are facing in the IAM space which is largely the same thing as what I am writing about in this blog so if you like this blog you may enjoy meeting me at Pulse as well.

[TIM] Contractor life cycle management in TIM

In IBM Tivoli Identity Manager the most common design for supporting the contractor lifecycle consists of a termination date on the user form and a "lifecycle rule" that basically disables/terminates any contractors who have termination dates in the past.

The update of the termination dates can either be handled by a helpdesk or it can be done directly by the managers as TIM has a very detailed and good user form access engine in the form of the ACL and views which means that you can actually grant access down to the individual field.

The lifecycle rule (scheduled task in OIM speak) is also very easy to implement as you can define an LDAP filter that gives you the users that are ready for termination.

Not very hard at all. At least not as long as your requirements are as simple and straightforward as this.

Contractor life cycle management

Most corporations have two general types of internal users.

The first category is employees or associates. The defining criteria of an employee is that they are present in the HR system. The HR system will tell us when a person joins the company, when a persons personal data changes or when the person leaves or gets fired. This is of course especially true if the HR system also feeds payroll. In most cases the IDM system will be directly connected to the HR system which will mean that the IDM system automatically will be notified about any changes.

The second categories are people that aren't directly hired by the corporation but rather are hired by another entity or they are hired by the corporation but in a different employment form. This means that this category of people are usually not present in the HR system. The users are still working for the corporation as consultants, contractors, contingent labor or partner employees. These users still needs access to IT systems so we need to somehow include them in the IDM system so that we can control their access.

If you are lucky the corporation actually has a database or some kind of application that tracks these users. In some cases it may not be one application but anywhere from a couple to a dozen applications. In this case you can just connect to these systems and you can handle these users the same way you handle employees.

In many cases there simply isn't any system that keeps track of the contractors so they need to be entered by someone directly into your IDM system. Not only do they need to be created but the contractor information also need to be maintained. Bearing this in mind what kind of lifecycle events do we typically need to support? In my experience these are the most important:

1. Creation
2. Update of personal data
3. Change of manager
4. Termination

Creation is usually very similar to creation of associate but there may be a need for an approval step and you may get problems with getting access to specific information about the user such as a unique serial number. Creation is usually driven by an employee that will act as the contractors manager.

Update of personal data is always really hard to get actually get up and running. In most cases personal information about a contractor user tends to get stale which may or may not be an actual problem for the IDM system. For example if a contractor gets married and changes their name in the name information in the IDM system should be updated but in many cases this information is never entered into the IDM system.

Change of manager is a critical use case if you are using the manager as a source of truth about the contractor. It is common that a contractor initially gets hired to work for one manager but later the contractor switches projects and ends up working for another manager. If you don't have updated and correct information about which manager each contractor reports to you will have a problem. Another problem that you will encounter is that the contractors manager may have left the company without first reassign their reports to another employee. It is often a good idea to include a check for reporting contractors in the termination process for an employee.

Terminations is of course just as critical as creations. We do need to remove the access for contractors that no longer is working for the company but there is nothing that motivates anyone to share the fact that a contractor has left with us. The contractor is gone so the contractor won't tell us. The manager is unlikely to tell us as the manager has nothing to gain.

The standard solution to this problem is to implement mandatory recertification of contractors. When a contractor joins the contractor gets a finite life span. Typically life spans is somewhere between 90 days to a year. Unless the contractor gets extended the contractor will be terminated once the end date is hit.

In most cases the extension is done by the contractor manager either directly in a self service interface or by calling the helpdesk. In order to ensure that the manager remembers to extend the user you usually implement a reminder process that sends out a number of reminder emails typically starting 30 days before the termination is supposed to happen.

If the user no longer works for the manager that the idm system has listed then you have a problem. This is especially true if the manager has left the company. In many cases it therefor a good idea to send the warning email to the user as well as the manager. The user is usually motivated to talk to their current manager who will then make sure that the IDM system is updated and that the user is extended.

Once you have these processes in place your contractors life cycles are in place the audit and corporate information security departments should be happy. Or at least happy enough to leave you in peace for a while.

Names

The names of persons is a subject that will impact a provisioning roll out in many ways. Some very upfront and other more convoluted. If you ignore the subject you may regret this once your system is live and the problems starts showing up. Names are not only used to populate the basic name fields but also as generators for things like logins and email addresses so by solving some of the problems up front in the feeds you can avoid a lot of issues in the target systems.

In this posting I will talk about static names while a later posting will discuss name changes.

If you are US based and you are rolling out an employee only system the name seeding is quite straightforward. The HR system usually uses the name that is featured on the social security card so that is the name that gets feed to you. For the last name this is usually straightforward but many people go by a different first name. Their real first name may be "Joseph" but everyone calls them Joe so they want their email to be joe.smith instead of joseph.smith. The solution to this problem is to have a preferred first name field and use that in case it exists.

Next problem comes in form of names that contains weird characters such as O'Malley. Normal solution is to just filter out any non a-z and A-Z characters from the feed. Hyphens are also usually allowed.

Outside of the US problems get worse. You have all kinds of strange characters in names and just filtering away all "strange" characters may not work very well. The happy Swede "Åke Öhlund" would not be so happy with the email "ke.hlund". What I have found is that you can actually support most languages by a simple translation table with about thirty entries that simply drops the umlauts and accent characters and turn them into the corresponding ASCII character. Our friend "Åke Öhlund" would usually like to get the email "ake.ohlund". Or at least he won't complain too loudly.

There a number of national or regional issues that I have run into over the years. If your system will cover these regions it is worth investigating if you will run into this specific issue or not.

In some parts of the world people have more than one last as well as first name. For example among expat Chinese in Singpore it is common that you have an official Chinese name and in addition a western name. A preferred last name as well as preferred first name solves this problem.

In Holland a lot of people tend to start their last name with van as in "van Fleet". There may be a request to generate email addresses with the van removed.

Germans like their formal titles and sometimes the Herr Doctors wants their Doctor degrees to be an integral part of their last name. Don't be surprised if you find "Schmitt, Dr" in the last name field in the HR feed. In severe cases you may even find "Schmitt, Dr, Dr" or "Schmitt. Professor Dr".

In Latin America people usually have two last names as your inherit one last name from your mother and one from your father.

Gartner Magic Quadrants

Gartner has published a new provisioning magic quadrant. In my opinion it is a quite interesting read.

One of the main points is that there has been a shift away from provisioning towards auditing, access recertification (attestation in OIM speak) and governance. Gartner uses the term IAI (Identity and Access Intelligence). The main driver here is the fact that provisioning projects are long, complex and painful while IAI projects are easier and quicker.

In my experience this is a correct observation. The real challenge in provisioning projects is that the provisioning process in an enterprise tends to be quite complex. In many cases the process isn't properly documented or there may be multiple different processes in different parts of the enterprise. The business analyzes work that is needed to document the process can be very time consuming and in many cases important points are missed.

An alternative is of course to simply adopt a "best practice" provisioning process but this requires a lot of political will and in many cases the complexities of the enterprise process is present for a reason.

On the other hand few companies have an established IAI process so adopting the "best practice" is relatively painless. This means that the time consuming step of documenting and implementing the current corporate business process can be almost completely skipped in an IAI project. The integrator can basically use whatever canned approach they happen to have handy which means that results can show up in weeks rather than months (or sometimes even years) which is the time scale you need for a custom provisioning implementation.

IAI projects are usually run to improve security and reach compliance but they can actually result in substantial operational efficiencies as well. In one IAI project we found 100+ user accounts for a quite expensive (1000+ USD per year license fee) application that really weren't needed. The lower license cost was a quite nice bonus for the customer but they actually were even more happy about the fact that we found 600 active remote access accounts that no one could explain who they belonged to.

OIM Howto: Interacting with Tivoli Directory Server

OIM has a quite large list of of connectors but sometimes you need to interact with a target system that lacks a standard connector. One such example is TDS (Tivoli Directory Server).

TDS is used as the internal directory server in TIM and can also be used as a standard corporate directory or as the user directory for the IBM Tivoli Access Manager. It is a standard LDAP v3 compliant directory so in theory you should be able to use any of the LDAP connectors (AD, eDirectory and Sun JDS). I would generally not recommend trying to use the AD connector as it includes a lot of functionality that addresses peculiarities in AD but either the eDirectory or Sun JDS will work fine.

The problems that may force you to write a custom connector will often have more to do with the lack of functionality of the standard connectors than incompabilities between TDS and the connectors. The eDirectory and the JDS connectors have basically gotten minimal updating during the last four years so compared with the AD connector their functional depth is quite limited. One area where you may see compability issues is in the handling of roles and groups.

You may end up writing some custom logic in JNDI to complement the functionality of the standard connector which really isn't very complicated if you have some basic Java programming skills. An example implementation of a JNDI based connector can be found in the JNDI demo tool