This is a general discussion of personality mechanics in role-playing games, where "personality mechanic" refers to any objective mechanic which advises or determines a player character's (PC's) voluntary behavior. There several types of personality mechanic.
The most common type is coercive, in that it requires the PC to act in certain ways. In general, the specifics are determined by the GM. For example, in the HERO system, a PC with a defined psychological trait (like "Overconfident") must make an EGO roll (i.e. roll under 9+EGO/5 on 3d6) in certain situations or he must act in a prescribed manner. The mechanics required certain traits (i.e. all PCs need to define a "Courage" stat), or it may be optional (i.e. each PC defines its own personality traits, if any). A subtype of this is social mechanics, where PC's behavior can be affected based on an NPC's social skill roll.
Another type of PM is representative. This is typically used in games where the personality of the character matters for some sort of supernatural powers, like magical mind control. For example, a PC might have a "Will" rating which is used to resist mind control or other spells. The rating might vary depending on PC actions. This Willpower rating is not used in regular decisions, only to resolve magical actions.
A third type is advisory. This is a personality mechanic which may be structured like coercive mechanics, except any results are totally under the player's control. i.e. The player decides if and when a roll is neccessary, and adjudicates for herself the results of that roll.
I would outline four potential pitfalls of personality mechanics in general:
The inner life of a character is the pattern of the character's thoughts and feelings which are not immediately expressed during play. For example, a player may slowly decide that his PC likes one NPC but dislikes another, based on a host of factors. This attitude may then have a wide influence on how the PC acts when they are around and in matters concerning them. Most personality mechanics will ignore this inner life. By imposing behaviors based on a random roll, the player might find that his PC instead is acting the reverse of how he thought. This forces the player to try to rationalize this after the fact, and reverses the previous pattern of behavior.
For example, suppose it is established that a given PC is a depressed alcoholic who is sometimes driven to drink. His drinking should really be patterned after his inner emotional state -- i.e. he drinks when he is depressed. However, the mechanic used doesn't factor in emotional state, but just makes it a random Willpower roll. This makes it impossible for the player to develop a coherent pattern of emotional swings.
A similar problem can occur with representative PM's, in that they can oversimplify the inner life into a few numbers. i.e. A PM represents a character's "willpower". When depressed, the alcoholic has lower willpower. However, the mechanic mixes up his manic phases of energy with his times of lucid, self-analyzing thinking. By representing both of these as the same effect (higher willpower), it can interfere with showing the difference of these two.
Most PM's attempt to reduce personality to simple numbers and keywords. i.e. A character takes the disadvantage "Greedy", but that keyword can mean many different things. People can be greedy for a variety of things, with varying motivations. The more personalized each stat becomes, the more you rely on judgement rather than on the mechanic.
Another problem is GM vs Player visions of the character. GM-moderated PM's establish the GM as the authority on what a character will do in certain situations. By taking this out of the hands of the player, it moves the responsibility. In general, any player will know far more about his character than the GM. The player has only his character to focus on. All other things being equal (which they might not be), the player should be better suited to be the authority on what the PC thinks and does. Taking away final authority, effectively says that the player is not to be trusted -- which can well be a self-fulfilling prophecy.
There are a number of differing thoughts on how to encourage role-playing. However, I have found broad agreement among players who tried removing PM's from their games that it is just as easy if not easier to encourage genuine role-playing. PM's have other uses, but they do not directly encourage role-playing.
So if the above are the pitfalls of personality mechanics, what are the strengths and benefits (if any)?
While they do not inherently encourage role-playing, they can serve to broaden and challenge role-playing. Sometimes you might want to break up patterns of behavior, because the process of justifying behavior is interesting. For example, one player might always tend towards playing the same character personality. This is not bad role-playing per se, but forcing a change might be interesting. Randomly rolled behavior can be entertaining and surprising, even if it is not neccessarily more realistic.
When using GM-moderated PM's, it is vital that you as GM realize that you are taking on a big responsibility. You need to communicate closely with each player to stay in tune with that player's vision of the character and his "inner life". Besides talking with the players in advance, you should stay aware of how the characters react to changing events. If you as GM can do this well, then you can work with the player to give more inner life to the character.
In addition, PM's can be used to communicate and remind players of aspects of genre. For example, even though it is not rational, characters in Arthurian myth often are overcome with emotion to the point of doing major acts seemingly without control. This exaggerated trope of the myths is reflected in the mechanics of Chaosium's Pendragon, which calls for rolls against various personality traits.
Lastly, PMs can give a sense of fair challenge to tasks which might otherwise seem arbitrary. For example, a player character could be the target of a seduction. Though the character is strong-willed and will try not to be tempted, even a good role-player may feel that there is a chance that he gives in.
Mechanic-less role-playing will certainly not produce "perfect" true-to-life behavior on the part of PCs. If you allow players to do what they like, they may well fail to properly represent their PC's fear (for example). However, it is important to remember that even though this is true, a randomized personality mechanic is not neccessarily better. The important question is: "Does this personality mechanic make for a better interpretation of the character than role-playing without it?" For example, a randomized more mechanic will lead to the PC running away at random times, which is not neccessarily more realistic than an fearless character.
There are certainly players and/or GMs who have found PMs to be of benefit to good roleplaying. On the other hand, there are players who have found them to be detrimental.