At its most basic, yes, programming languages are just a "convenience". But it is actually more than that. It's abstracting ideas from mechanics such that those ideas become clear and more meaningful.
To illustrate, I'll use a "programming language" that is much older than the concept of mechanical computers: algebra.
Yes, basically algebra is just a convenience for mathematicians to write equations in shorthand. But it is more than that, it makes concepts in mathematics clearer and easier to discuss. Imagine being a mathematician in Pythagoras's time who did not have access to the "programming language" called algebra. Working in mathematics involves reasoning with the "assembly language" called "human speech":
The area of a square whose sides are constructed from the hypotenuse
of a right angle triangle is the sum of the area of two squares constructed
from the remaining sides of the triangle.
This was actually how the ancient Greeks reasoned in mathematics. And it was tedious. Then the Arabs came and invented symbolic manipulation and implemented it in the "programming language" called algebra. Now you can discuss the Pythagorean theorem clearly with:
2 2 2
a = b + c
Much easier to manipulate and use. Even though both methods of reasoning are technically the same using the algebraic form to find the length of a side is easier:
_______________
/
/ 2 2
b = V a - c
Imagine trying to figure that out without algebra.
It's similar with programming languages. If you just work in assembly or even C you will never consider concepts like closures, continuations, monads, object orientation etc. Some of these concepts are fundamental to computer science but is very alien to computer hardware. To work comfortably with these concepts you need to abstract the ideas away from the implementation - thus programming languages are born.