Juggling Vacation Time with Work Responsibilities

I am approaching my ten year anniversary with my company, which garnered me a 4th week of vacation this year. I have the least amount of tenure on my team, which I joined 2.5 years ago. Everyone on my team is at 4 weeks of vaction (with the option to rollover a max of 5 days). I recently spoke to my husband regarding using a vacation day for a long weekend, and he was very discouraging of my using it, stating that my taking time off could make my employer think that I am not needed in my role and that my position could be eliminated. A few caveats: 1) my company did a lot of restructuring at the end of the year and offered retirement packages to avoid downsizing - although some positions in other departments were eliminated. 2) a member of my team is on medical leave currently and she and I are the only people who know how to do certain tasks. Between that and my boss' complimentary assessments of my work at our fortnightly one-on-one meetings, I feel reasonably secure in my position. 3) My father is in in late stages of life ending disease and depending on the speed of his continued decline, I may need to use more leave time to spend time with him and my mother who live several hours away. (Another point my husband made about why I shouldn't use vacation leave; although my father's condition could mean that he has a year or 5 years or somewhere in between). 4) I am extremely conscientious about scheduling my time off, checking my colleagues' calendars to make sure that my absence would not cause our department to be shorthanded. If I desire to take a day off for pleasure and two colleagues are already scheduled to be off, I will select another day. 5) my husband owns his own business; therefore, he works considerably longer hours than he would work if he were employed by someone else. He has been self-employed for 20 years now. So I'm not entirely sure about the accuracy of his perception of the world outside his small business given that length of absence from it. Compared to my colleagues, I use the least amount of vacation/sick/work from home time by a considerable amount. Each year, I carry over the maximum days allowed (and have even sacrificed vacation days in the past because projects necessitated that I be at work). But now, my husband's words have wormed their way into my brain, and I am apprehensive about using my vacation time. Is his claim something that I should take into consideration? Or is he simply overthinking things in regards to caveat #1? I would assume that managers want their employees to utilize vacation, so they can be refreshed and ready to work instead of burned out and overwhelmed. My boss takes several vacations a year as does his boss, so it seems that management is of this mindset.