Neil Diamond has been holed up at the Arc Angel, the L.A. studio he's called home for more than three decades, working on his new album, 'Home Before Dark.' Before he plays it across the U.S., Diamond has an unlikely engagement: A two-night appearance on 'American Idol.' The show will introduce the songwriter behind the Monkees' 'I'm a Believer,' UB40's 'Red, Red Wine' and Urge Overkill's 'Girl, You'll be a Woman Soon' (not to mention his own 'Sweet Caroline' and 'I Am...I Said)' to a new generation of music devotees.

The release of the sublimely intimate, Rick Rubin-produced 'Home' -- which features a duet with the Dixie Chicks' Natalie Maines -- reminds us where else fans should be able to catch a glimpse of Diamond: In the Rock & Roll Hall of Fame. Spinner caught up with the legendary singer-songwriter to talk about everything from Frank Sinatra to the mysteries of songwriting.

How was working with Rick Rubin for the second time different from the first?

It was pretty easy both times, although it was probably more [a process of] getting to know each other the first time. Maybe I was a little more nervous the first time working with these musicians, the first time working with Rick. You put yourself in somebody's hands and you really don't know what they're going to do. But as it progressed, it became apparent that it was going to be comfortable and it was going to be good. This one was a lot easier. There were less songs. I knew the musicians. We'd done an entire album together. You get to know people pretty good working that close.

Did that relaxation influence the album?

It did influence the writing, but I'm not really sure how. See, I hadn't written an original album .... I started really with the 'Tennessee Moon' album, co-writing all the songs. It was intentional to work with a lot of different writers in Nashville. I had a lot of fun. That started to get the barnacles and rust off of my writing chops. Then, 'Three Chord Opera' was maybe all mine. That continued that process of the rust falling off. And '12 Songs' took me just a little bit further. I was flying with this one. It's about as good as I can get as a writer. I was in the groove. I knew what I wanted and when I heard something that I liked, I knew when to follow up on it.

Was there anything that surprised you that came up in the songs?

A lot of it's stream of consciousness and only in a couple of songs did I actually know what I wanted to say. Usually there was a title and a melody, so that's the concept. For a songwriter, the title should be a summation of what you want to say, or should reflect it. But they all made themselves apparent at some point or another. I didn't know what a lot of them were about and they showed themselves as the writing and recording continued. I don't know how to compare it to anything. You write, so you find a phrase -- an idea that just comes to you. You like it, you want to build on it, explore it, see where it takes you. It's a signpost. You can be thrilled by it and how you use it. What does it mean? And does it have to mean anything literally, or can it just be an emotional statement? It defies description. It's work. You just jump in and learn how to swim each time. I've been doing it since I was 16, and I still couldn't possibly describe what the process is. I sit down, try and find something that touches me or moves me, build on that and make it into a fully-realized musical statement.

Do you remember the first record that stirred something in you?

For my 16th birthday, I got two Weavers albums from a group of friends: 'The Weavers on Tour' and 'The Weavers at Carnegie Hall.' I learned how to play every one one of the songs. I had just started taking guitar lessons and I loved them. Those were the first that I can remember.

What music have you heard recently that you connected with?

I don't listen to a lot of music when I'm writing and, unfortunately, I'm writing most of the time. I don't like to pollute my brain with other people's stuff. I miss out on a lot. As soon as I finished writing this album, I immediately wanted to hear what music was out there. The last thing that really knocked my socks off was Johnny Cash's ['Hurt']. That was powerful. That's the kind of stuff that an artist and writer really aspire to -- to have that kind of power.

It's interesting that you said this album was stream of consciousness because it feels like it has a lyrical theme.

Because of my personal experience this last year-and-a-half, whatever I wrote would have some kind of theme. But it wasn't intentional. I just wrote the songs that I was able to find -- to dig up from somewhere in my gut, mostly, and somewhere in my brain. I'm very close to it. I'd have to listen to it in a year from now and tell you what it's about. That's usual with my records. When I finish them I won't listen to them for a year or two, just because it's like something you get off your chest and don't want to have to revisit it for a while. When things have calmed down, the dust has settled, you can listen to it and say, "Wow, I like that a lot," or "Missed on that one. I could've done it differently or better." So, I'm too close to it now. It's just being born. I'm just hoping I like it.

What were your thoughts when you listened to '12 Songs' a year later?

For the most part I thought it was good, but that was such a heavy load. To work on so many songs at the same time was way too much. It was a burden that I was glad to leave behind They're all burdens, but this one was more fun and more variety. I like what we have so far. We'll see. But I do trust Rick.

Rick had long-expressed interest in working with you. Did it make working with him easier, knowing that he was approaching this from a fan's perspective?

I never got the impression that he was a big fan. I wish I would've know that. It would've made it a lot easier for me. Rick is not the kind of guy that pats you on the back a lot. He lets you do your thing, and then he's a very severe editor. I felt I had to establish myself and my credentials in his eyes from the get-go.

Was there anybody you've met where you felt the need to keep your fandom in check?

When I did the Nashville album, Chet Atkins played on one of my songs and he did the video, as well. Chet Atkins was like god to me when I was a teenager learning how to play guitar. I could not accept that Chet Atkins was playing on my record and he was guesting on my TV show. I've worked with a lot of people, but having Chet on my record was awesome.

Do you have a favorite cover version of one of your songs?

Oh, yeah. That would be Frank Sinatra's version of 'Sweet Caroline,' which he did with a big band. That would absolutely be my favorite. He did it his way. He didn't cop my record at all. I've heard that song by a lot of people and [there are]a lot of good versions. But Sinatra's swinging, big band version tops them all by far.

What are your parting thoughts on 'Home Before Dark'?

I'm happy to have it under my belt and to have survived it.