When we say landlords are bad, it's not really about the individual people so much as it's about the system as a whole. Ideally, the human right to housing should be guaranteed for everyone, along with the right to be cared for in retirement. How many elderly people don't own their own homes, and have rent to pay as an additional expense making it harder for them to retire? Sure, landlordism can provide a source of income for people who can't work, but for every case of that, there's another case of someone who can't work who doesn't have the privilege of owning a home, such that the existing system makes them even more desperate. So logically, it doesn't really make sense as a justification.
Cases like this should be considered when we're looking at how best to implement our ideals, but not for determining our ideals in the first place. The just thing is that everyone should have a secure place to live. That's the ideal. In implementing that ideal, we should consider that houses currently are used as a form of investment and many people simply use them that way without a second thought, because of social norms. If we simply seized and redistributed everyone's properties tomorrow, some people like your aunt would be disproportionately affected, compared to if they had invested in stocks that can be just as unethical. It would probably still be better for most people than doing nothing, but we ought to craft policy in such a way that we're not trolley probleming it (except regarding the people at the very top, for whom it's unavoidable), but rather such that it provides benefits while harming as few people as possible.
When society is organized justly and the wealth of the people on the top is redistributed, there will be enough to go around that everyone ought to be able to benefit from it. Therefore, it shouldn't be a problem to compensate small landlords for their properties and ensure that they aren't harmed by any changes in policy.