The question is, what’s going on here? The obvious answer is sexism and misogyny, but the fact is, that’s more an effect than cause. It’s really not about representation. It’s about spreadsheets and demographics and market segments.
Let’s take Disney as an example, since they own most of the major properties in question. Within the Disney corporate structure, there are many departments, and each of them is expected to generate revenue and contribute to the bottom line. The various production divisions (Feature Animation, Motion Pictures, TV, etc.) make the movies, cartoons, TV shows, and other content, and the Licensing department sells products based on them. The goal for each department is to maximize sales.
For the entertainment department, their approach is to include something for everyone; humor, action, romance, drama, to cast as wide a net as possible. The toy business takes the opposite approach; they go for specialization and segmentation, marketing different lines of products to carefully-defined audience groups, such as young children, boys age 8-12, girls age 8-12, teen girls, teen boys, and adult men and women, also broken down into age groups. They are very aware that men over 50 don’t buy the same things as men under 30, and neither of them buy the same things as women of any age. And vice-versa.
Until very recently, this specialization theory included some universally held beliefs about what boys and girls wanted and would accept. Most of these focused on what boys would reject. For most of the 20th century, social boundaries dictated that boys were expected to firmly reject anything that was seen as for younger children (“baby stuff”) or for girls; letting it be known that you like either of these things was an invitation to harassment, condemnation, and rejection, and boys understood it very well. Beginning in the 1980s, toy manufacturers grabbed this notion and made it a rigid rule, and exacerbated the problem by adding gender signifiers (pink and purple for girls, black and red for boys) to formerly gender-neutral toys.
For some time, there’s been a tug-of-war within Disney between the licensing and production departments. Some years ago, Disney licensing deliberately created the “Disney Princesses” line as a way to lean into the gendering of the toy aisles and sell a lot of stuff to little girls. They also cherry-picked Tinkerbell out of the Peter Pan story and created the “Disney Fairies” brand. These branding programs were hugely successful, but they were directly in conflict with the studio’s “our movies are for everyone” philosophy. In marketing the toys, they also accidentally re-branded most of Disney’s back catalog as “girl movies.”
As licensed merchandise became a bigger part of the revenue pie, the marketing segmentation policy was imposed, at least to a degree, on the production end. The unfortunate result is that audiences no longer look at movies like Beauty & the Beast as children’s movies or family films; The Little Mermaid is a “princess movie” and The Lion King isn’t. This means that if Disney wants to attract the whole family to a movie, it can’t look even slightly like a “princess” movie; if it does, it has to have a bland gender-neutral name and an ad campaign that hides the female characters. For examples, see Frozen’s ads that focused on the snowman and reindeer, and Tangled’s Flynn-centric campaign. Both of these films got their titles because of the paralyzing fear that “Snow Queen” and “Rapunzel” are too close to “princess.”