How do I prevent window.pageYOffset from resetting on render?

I have a react component where I want to change the header based on the scroll event. I’m attaching an event handler and based on scroll position and toggling a display class to hide ro show the desired elements.

Where I’m having trouble is, there seems to be some glitchy behavior when my component tries to re render. I have an example in codesandbox below.

import React from "react";
import "./styles.css";

export default function App() {
  const [scrollY, setScrollY] = React.useState(0);
  React.useEffect(() => {
    const handleScroll = () => {
      console.log(window.pageYOffset);
      setScrollY(window.pageYOffset);
    };

    window.addEventListener("scroll", handleScroll, { passive: true });
    return () => window.removeEventListener("scroll", handleScroll);
  }, [setScrollY]);

  const scrolled = () => scrollY > 40;

  return (
    <div className="App">
      <div className={`header ${scrolled() ? "d-none" : ""}`}>Header Main</div>
      <div>
        <div className={`header-secondary ${scrolled() ? "d-none" : ""}`}>
          Header Secondary
        </div>
        <div className={`header-scrolled ${!scrolled() ? "d-none" : ""}`}>
          HeaderScrolled
        </div>

        <div>Scroll Position: {scrollY}</div>
        {[...Array(100)].map((e, i) => (
          <div>
            <div className={scrolled()}>{`SCROLLING`}</div>
          </div>
        ))}
      </div>
    </div>
  );
}

My code sandbox: https://codesandbox.io/s/wizardly-saha-0oklr

If you notice I have my hide/unhide compnent condition set at 40px. When you scroll slowly around 40px the header will snap back and for some reason the window.pageYOffset will reset to 0. I can’t figure out why this is?

If you scroll past fast enough it doesn’t matter but right around where I toggle the display class there is some odd behavior.

EDIT: Updated Example
Effectively what I need to do is have a smmoth transition from Header Main to Header Secondary. I can’t really change styling on Header Main because I don’t own that part of the product.