I'm a new admin to a company I started with in March.

I'm a new admin to a company I started with in March. A day porter was hired the same day I was from a company he was on contract with while working at the company I was hired at. I noticed he has not been fulfilling his job duties, been absent frequently. I spoke to my manager about it and she dtstrd she would meet with him & go over his job description. There are days our manager is off & works from home. Things aren’t been done, so I step in to take care of things, which are his job duties. He stated to our manager that I’m managing him. I don’t want things to fall on me & my colleague. What advice can you give me to handle this issue properly?