TL;DR If you want a quick way to gauge whether or not students understand the concept of potential energy, you can ask them to explain the origin of The World’s Most Annoying Negative Sign. It is merely a consequence of the very definition of potential energy, which is almost never explained coherently.
In all of my undergraduate and graduate physics courses, I don’t remember ever having talked about the concept of “system” at all. In the Matter & Interactions curriculum choosing a system takes a major role because all of the fundamental physical principles apply to a physical system. If you don’t choose a system, you can’t do physics.
What is a “system” anyway? Definitions vary across physics, engineering, and chemistry, but I explain it as “the particular part of the Universe we’re immediately interested in for the duration of the current discussion.” A single particle can be the system. The Universe itself can be the system. A set of particles can be the system. Particularly relevant to this post is the fact that a field (like an electric field or magnetic field) can be included in the system.
I will use terminology from Chabay and Sherwood. So, consider a system consisting of a particle with charge 
and an electric field 
. Writing the Energy Principle in its most generic form gives
![\[ \Delta E_{\textup{sys}} = W_{\textup{total}} \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-9e3600c07c5e9ba902591de3334e21a4_l3.png "Rendered by QuickLaTeX.com")
meaning that any work done on the system will change the system’s total energy. Let’s expand the right hand side into external work and internal work to get
![\[ \Delta E_{\textup{sys}} = W_{\textup{ext}} + W_{\textup{int}}. \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-c067e0922dbe72991ba83e68f091145c_l3.png "Rendered by QuickLaTeX.com")
We can always strategically choose our system so there are no external forces doing work on the system, so the first term on the right hand side is merely 
. Let’s rewrite to account for that, and let’s expand the left hand side to get
![\[ \Delta E_{\textup{rest}} + \Delta E_{\textup{kinetic}} = W_{\textup{int}}. \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-9a880d1fdb779f253c53ee377d9e751f_l3.png "Rendered by QuickLaTeX.com")
where we have accounted for the fact that inside the system, assuming we’re dealing with particles, we can have a change in rest energy (thanks to nuclear phenomena) and a change in kinetic energy (particles usually move around). But in this case, our system also includes an electric field, and that field and the particle definitely interact with each other. There’s nothing of a nuclear nature going on here, so 
is . A charged particle will usually accelerate in the presence of an electric field so there’s probably a non-zero 
present. But how do we account for the interaction between the particle and the field? That interaction is internal to the system, and the field will definitely do work on the particle. Therefore, let’s move the 
term to the left hand side to get
![\[ \Delta E_{\textup{kinetic}} - W_{\textup{int}} = 0. \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-34c5dac6a8fbf91db0522c56d47cbdb2_l3.png "Rendered by QuickLaTeX.com")
There it is! It appeared so quickly and so naturally that you may not have even noticed. There’s The World’s Most Annoying Negative Sign in front of the second term on the left hand side. Now, as quickly as it appeared, we’re going to make it go away again by performing nothing more than a definition of a new quantity that absorbs The World’s Most Annoying Negative Sign. Watch closely, because it happens quickly.
![\[ \Delta U_{\textup{e}} \equiv -W_{\textup{int}} \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-f4d6dea93983f6deda907bb6311db2eb_l3.png "Rendered by QuickLaTeX.com")
Now let’s write the Energy Principle to use this new definition to get
![\[ \Delta E_{\textup{kinetic}} + \Delta U_{\textup{e}} = 0. \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-79e13bf3a1c27ee5791d7ad34c94e8cc_l3.png "Rendered by QuickLaTeX.com")
That new symbol on the right hand side has a name; it’s the “change in electric potential energy of the system” and note that it’s merely a symbol 
to hide The World’s Most Annoying Negative Sign! It’s just that simple.
The World’s Most Annoying Negative Sign usually appears in the expression for the change in electric potential:
![\[ V_{\textup{b}} - V_{\textup{a}} = \int^{\mathbf{r}_{\textup{b}}}_{\mathbf{r}_{\textup{a}}} - \mathbf{E}\bullet \mathrm{d}\mathbf{r}. \]](https://tensortime.sticksandshadows.net/wp-content/ql-cache/quicklatex.com-f07b7519834f332a2d5980cc7bc6f2f0_l3.png "Rendered by QuickLaTeX.com")
Sometimes it is inside the integral as shown here, and sometimes it’s outside the integral. To confuse matters, it’s sometime omitted altogether!
I remember combing through various sources in both undergrad and grad school looking for consistency. Different authors used different conventions and I was perpetually confused. That’s precisely why I named it The World’s Most Annoying Negative Sign. It wasn’t until I first read Chabay and Sherwood that I saw how logical the sign’s presence is, and that’s it’s nothing more than a consequence of energy bookkeeping in the Energy Principle. Specifically, everything on the left hand side of the Energy Principle is inside the system, and everything on the right hand side is outside the system. This simplicity requires a change in sign of a quantity moved from the right hand side to the left hand side, and then a new symbol and name are invented to quickly hide the negative sign again. It all makes sense now. I think the most important part of this story is that The World’s Most Annoying Negative Sign is a consequence of including this abstract thing we call and electric field as part of our system. Fields are real, physical entities and not just mathematical devices. A charged particle can interact as legitimately with an electric field as legitimately as it can with another charged particle, and accounting for this requires The World’s Most Annoying Negative Sign.
Am I missing something if I’ve always been perfectly happy to think of it this way:
http://hyperphysics.phy-astr.gsu.edu/hbase/pegrav.html#pen
Either think of it either as: the force is the downhill gradient of the potential (which brings in a minus sign) or that: the change in potential is the work you do against the force (which brings in the minus sign due to N3L)
Thank you for the comment. I prefer to carefully choose my system and then look at whether or not the electric field is inside my system. If it is, only then can I meaningfully talk about the change in the system’s potential energy, which by definition is the opposite of the work done by internal forces (thus the negative sign). If the electric field is outside my system, then I can’t meaningfully invoke the concept of potential energy and I must use the concept of work. Potential energy and its change is a property of the system and not, say, a charged particle that happens to experience the electric field and thus experience a force, and we can calculate the associated work. Of course, the final number will be the same either way, but I’ve seen too many sources sometimes write the negative sign and sometimes omit it and baffle students (like me when I was a student). The system approach makes everything consistent and logical.