Dude, I said English was harder. Seriously, try to keep up! I just said it's not much harder and comes with the benefit of people actually speaking it so that learning it isn't a waste of effort.

Further, Esperanto is ignored because it's not much easier than natural languages to huge swathes of the world's population, but at least has the benefit of being utterly useless to learn.

Learn a few languages from places that aren't Indo-European ones. Learn how you can have grammars with little to no declension, for example: no verb tenses, aspects, voices, genders, cases ... not even declining by count. Then consider:

1. Esperanto has almost all of these alien-to-many concepts; and,
2. While it is true that it is more regular in these than in natural Indo-European languages, the latter have the benefit of actually having speakers: the purpose of learning a foreign language is met: communication.

On top of this:

1. Esperanto has a consonant-heavy phonetic inventory, making its pronunciation hard for a lot of speakers of other languages. (It is painfully obvious that Zamenhoff was Polish, let's put it this way.) Too it is very bizarrely irregular (though it's not so bizarre once you check out Zamenhoff's native dialect and its consonantal inventory...). Lest you think this isn't a problem, most native languages in the world rarely present more than "consonant+vowel" structures, so strings of consonants are absolutely horrendously difficult for them. (Even saying "string" is hard, and that's mild compared to some of the atrocities of ~~Polish~~Esperanto.
2. Esperanto uses a system of affixes (pre- and suf-) to words to modify word forms and attach meanings. This is a difficult concept for speakers of languages like Mandarin, say, to comprehend (where word forms are notoriously vague and grammatical particles are used in place of affixes to accomplish many of the same things). Further, Esperanto assumes that a) word forms are universal, b) that the categories in those languages that have them are the same, and c) that even when the categories are the same individual words are categorized similarly across languages. Yet in English "angry" is an adjective. In other languages it is a verb. Fancy that!
3. Esperanto has the single most useless feature of any language: gendered declensions. (And, naturally, just to add icing to this cake, the default is masculine.) Zamenhoff had the chance to remove the single most useless feature of a language from his grammar ... and didn't. Flipping FARSI managed to do this, a natural language in the Indo-European family, but a constructed language had to keep this vestigial nonsense?! Again, gendered grammar is not even slightly universal and makes the language difficult to learn for people coming from sane languages.
4. Esperanto's lexical inventory is gloriously East European for the most part, with random slathering of Romance-language vocabulary generously applied. So, you know, using as a basis words from a small geographical region instead of words from around the world. Where are the Chinese roots? The Arabic ones? The roots from various African languages? There aren't any. Thus it is pretty much equally difficult for a Chinese(or Arabic(or, say, Swahili))-speaking student to learn the lexicon of an actual language spoken by actual people instead of a toy language spoken by basically nobody.
5. What is a subjunctive? What is an infinitive? What is a participle? These are concepts that are very much Indo-European. Speakers of languages outside that family (which is checks notes most people) have no idea what one or more of these are. So that's three alien grammatical concepts right off the top of my head in Esperanto's grammar, and while sure it's more regular (FSVO "regular") than in natural languages, it's the conceptual barrier that is hard to breach, not the rote memory work to learn them once you've grokked the idea. So again, slightly more difficult to learn a natural language, but even a natural language with as low a speaker count as Basque will give you about as many people to talk to as does Esperanto while the Big Name™ languages will give you multiple of orders of magnitude more. Each.
6. Esperanto assumes that notions of "subject", "object", and "argument" are linguistic universals. They aren't. This makes Esperanto's twee case structure with its cute little suffixes actually fiendishly difficult to learn for speakers of languages that mix agents, experiencers, and patients in ways different from the Indo-European majority. (Don't know what agents, experiencers, and patients are? Maybe you should crack open an inventory of linguistics before talking about how "easy" a language is to learn...)
7. Why are there plurals in Esperanto? Why decline for number at all? Plenty of languages don't and it works just fine. OK, so for whatever reason you think plurals are necessary: WHY THE HELL DOES ESPERANTO ALSO HAVE COUNT/VERB AGREEMENT!? That's just bizarre even in many languages that have retained the unnecessary concept of a plural!
8. Personal pronouns. Ugh. There's first person singular and plural (but no way to distinguish between inclusive and exclusive in the latter case). There's second person with no ability to distinguish singular and plural (because consistency is for whiners!). There's gendered (🙄) singular third-person, but non-gendered (let's be honest: default-masculine) third-person. And then there's a weird one (oni) that means one. Or people. Because screw making sense! Why are there gendered pronouns at all!? They serve no useful purpose; many languages (including Farsi, the language of Iran(!)) eschew them completely, and others (e.g. Mandarin) only distinguish them in writing (and that itself is a very recent cultural import!).
9. Articles. WHY IS THERE AN ARTICLE IN ESPERANTO!? And why only one!? You've eliminated all the other articles, take that final step dammit! Join the majority of world languages which don't bother with these vestigial adverbs!

And I'm out of steam already. There are a whole lot of hidden linguistic assumptions in Esperanto that are alien to language speakers from outside of the Indo-European milieu, or difficult for such speakers to actually perform. To someone in steeped an Indo-European linguistic environment these are invisible. They're "natural" or even "logical". But they are absolute tongue-twisters and conceptual mountains for those coming from outside of those environs. And if you're going to climb those conceptual mountains and twist your tongue in service of these phonetic horrors, where do you think it's best to expend your efforts:

1. On a fantasy football league language that has maybe a million speakers world-wide (and that's being generous!); or,
2. On a natural language that's a little bit more difficult but gives you access to ~1 billion native speakers and ~200 million secondary speakers (Mandarin), ~475/75 million (Spanish), ~400 million/~1 billion (English), 350/250 million (Hindi), or even 50/26 million (Hausa)?

If you're sane and value your time, you pick literally almost any natural language in the world for better return on investment, even though it may, in the case of some of those (coughIndo-Europeancough) languages, be a little bit more difficult than Esperanto. (Yes. A little bit.)