Should I believe him??

MMkay, so my boyfriend gets off work at 5 and his boss drops him off at my house... He doesn't have a key with him. Our next door (FEMALE) neighbor that HE knows but I don't know, is getting home at the same time..They say hi to each other and he says "I think I forgot my key" and she invites him over for a beer while he's waiting... I come home 40 minutes later, get our baby out of the car and lug him inside and then sit down on the couch... He walks in the door a few minutes later... He KNOWS it will piss me off that he was over there, so he tells me he's been out walking around. I ask him, "What have you really been doing?" and he says blah blah blah her house blah blah blah... He has a history of lying when it comes to drug-related issues, but I don't think he's EVER cheated. He says nothing happened he was just waiting, which sounds completely believeable, but IS IT???