The German international made 151 appearances for the Gunners and won the FA Cup twice after joining from Valencia in 2016.
"We thank Musti for his contribution at Arsenal which has spanned four-and-a-half years, more than 150 appearances and a prominent role in two FA Cup wins," said technical director Edu.
"Everyone associated with the club wishes Musti and his family the very best wishes for the next chapter in his career.”
Mikel Arteta added: “Musti has been a key player for us for a long period of time.
"He played a very important part in my first season as manager and I’d like to thank him for his professionalism and for always being ready when called upon.
"Everybody at Arsenal wishes Musti well at Schalke.”
Arsenal also thanked Mustafi for his contribution to the Islington community during the coronavirus pandemic.
"We are also thankful to Shkodran for the part he played off the pitch," said a club statement.
"His donation of more than 16,000 meals helped vulnerable residents in our local borough of Islington during the height of the coronavirus pandemic, providing them with an essential food service they otherwise wouldn’t have received.
"Everyone at Arsenal thanks Musti for his contribution to the club. We wish him and his family all the best for the future."
Mustafi made a strong start to his Arsenal career after arriving in a deal worth a reported £35m, going unbeaten in his first 22 games for the club.
But, the World Cup winner came in for criticism on a number of occasions for high-profile defensive errors.
He will play alongside Sead Kolasinac at Schalke after the Bosnian joined the club on loan from the Gunners last month.