Noah Baumbach made a bit of a splash with his excellent 2005 feature The Squid and the Whale, which was steeped in his own family history, and darkly funny, but never shying away from pain or truth. Everything seemed to click on this movie. Unfortunately his next feature, Margot at the Wedding (2007), failed to repeat the trick; this one came across as agitating and prickly, with characters that never connected and all-too-obvious dialogue and symbolism (a dead tree?). Baumbach's new Greenberg seems to fall somewhere in the middle. It's a tough film to read; it's definitely irritating and off-putting, but it also seems to come from a place of genuine anguish.

Part of the film's success -- and trouble -- is lead actor Ben Stiller. Stiller is perfect for this kind of selfish misfit, and Baumbach reels him in before any typical Stiller slapstick can take over. But he also keeps himself at a little distance, a little defensive. He plays the title role, Roger Greenberg, who winds up in Los Angeles, housesitting while his brother and brother's family vacations in Vietnam. Roger has apparently just been released from some kind of mental hospital, and at age 41 has decided to "do nothing for a while." He passes the time doing some carpentry and writing complaining letters to corporations. While in town, he catches up with an old friend, Ivan (Rhys Ifans), with whom he was once in a band and was once inches away from a record deal. Roger goes to a party and sees his old girlfriend, Beth (Jennifer Jason Leigh), who is newly divorced. He begins to imagine getting back together with her.

Roger's brother's personal assistant, the 25 year-old Florence (Greta Gerwig), also turns up, and of course her services are available to Roger as well. She's a bit of a nut, and perhaps a tad insecure, but far from incompetent. Her trademark line comes while driving, speaking to drivers in the next lane as if they could actually hear her: "Are you gonna let me in?" Florence and Roger begin to see each other for a couple of awkward, indecisive dates, and then are forced into each other's company when the family dog, Mahler, gets sick. Their scenes together make up an odd little mating dance of ins and outs. Roger makes a move and Florence responds, but then Roger backs off and picks fights with her. He's attracted to her, but guilty about it, and perhaps he thinks he's not good enough for her, or too old for her. Florence, on the other hand, seems genuinely moved by him, and perhaps drawn to his wounded, distant quality.

The characters are there and the performers are willing, but Baumbach provides them with the same type of quirky dialogue he helped cook up for Wes Anderson's films The Life Aquatic with Steve Zissou and Fantastic Mr. Fox. In other words, characters speak at each other rather than with each other; each character responds in some way that's 180 degrees away from the previous comment. When Anderson handles this dialogue, it's funny and even revealing, but Baumbach can't quite get inside it. The words just flop out and land with a thud.

This Anderson influence lingers on one end of Greenberg and the "mumblecore" influence is very obviously at the other end, especially with the presence of Gerwig (Hannah Takes the Stairs, LOL and Baghead) and an appearance by Mark Duplass (The Puffy Chair, Humpday). The trademark of "mumblecore" is the quality of the dialogue, which comes from the mouths of overeducated, underemployed, bored twentysomethings, and sounds eerily, uncomfortably realistic and rambly. Baumbach's style is too overly written and carefully prepared to capture this organic aimlessness, but it's also not careful enough to get into Anderson territory. Greenberg tries to be an aimless movie about aimless characters but the road map is both too carefully marked and too deliberately obscure.

Which is not to say that it's a complete dud, either. Baumbach sometimes comes up with some wonderful moments of self-reflection, mainly without the use of words at all. In one scene, Roger finds himself at a birthday party surrounded by kids and old, alienated friends. He tries to connect with a few of them and fails and finally just stands in the middle of it all, his brain drifting off to some secret place of pain or comfort, as Baumbach's camera simply draws backward from him, followed by an unexpected, snap cut in the middle of a music cue. Baumbach also dabbles and doodles with lonesome, urban imagery like a running garden hose snaking through a swimming pool, or one of those weird "air dancer" men that you see on car lots.

I was also quite moved by the performance of Rhys Ifans, who carries Ivan's emotional baggage much more quietly and with more dignity than any other character. His little moments of trying to connect with Roger come from a deeper place than Florence's and they seem to build from scene to scene; the men grow more and more comfortable with one another, and as they do, they become more honest. Their pain comes to the surface. His finest moment comes during the movie's third act when the men finally have their big blow-up fight. Ivan walks out, but returns for one more comment: "this is a small thing, but..." And his delivery is a devastating blow.

Perhaps Greenberg grows more comfortable with itself as it goes. It's less eager to please and more willing to revel in some real-life emotions rather than indie-movie quirk. Its final line is the most delicious touch. I won't give it away, but it's the movie's first attempt to nail down Roger's character and give him his purpose. It comes late -- both in the movie and in Roger's life -- but at least it's there.