The idea of representing the processes of calculus, differentiation and integration, as operators
has a long history that goes back to Gottfried Wilhelm Leibniz. The mathematician Louis François Antoine Arbogast was one of the first to manipulate these symbols independently of the function to which they were applied.
A rigorous mathematical justification of Heaviside's operational methods came only
after the work of Bromwich that related operational calculus with
Laplace transformation methods (see the books by Jeffreys, by Carslaw or by MacLachlan for a detailed exposition).
Other ways of justifying the operational methods of Heaviside were introduced in the mid-1920s using
integral equation techniques (as done by Carson) or Fourier transformation (as done by Norbert Wiener).
A different approach to operational calculus was developed in the 1930s by Polish mathematician
Jan Mikusiński, using algebraic reasoning.
Norbert Wiener laid the foundations for operator theory in his review of the existential status of the operational calculus in 1926:
The brilliant work of Heaviside is purely heuristic, devoid of even the pretense to mathematical rigor. Its operators apply to electric voltages and currents, which may be discontinuous and certainly need not be analytic. For example, the favorite corpus vile on which he tries out his operators is a function which vanishes to the left of the origin and is 1 to the right. This excludes any direct application of the methods of Pincherle…
Although Heaviside’s developments have not been justified by the present state of the purely mathematical theory of operators, there is a great deal of what we may call experimental evidence of their validity, and they are very valuable to the electrical engineers. There are cases, however, where they lead to ambiguous or contradictory results.
The key element of the operational calculus is to consider differentiation as an operator p = d/dt acting on functions. Linear differential equations can then be recast in the form of "functions" F(p) of the operator p acting on the unknown function equaling the known function. Here, F is defining something that takes in an operator p and returns another operator F(p).
Solutions are then obtained by making the inverse operator of F act on the known function. The operational calculus generally is typified by two symbols, the operator p, and the unit function1. The operator in its use probably is more mathematical than physical, the unit function more physical than mathematical. The operator p in the Heaviside calculus initially is to represent the time differentiator d/dt. Further, it is desired this operator bear the reciprocal relation such that p−1 denotes the operation of integration.
In electrical circuit theory, one is trying to determine the response of an electrical circuit to an impulse. Due to linearity, it is enough to consider a unit step:
Rota, G. C.; Kahaner, D.; Odlyzko, A. (1973). "On the foundations of combinatorial theory. VIII. Finite operator calculus". Journal of Mathematical Analysis and Applications. 42 (3): 684. doi:10.1016/0022-247X(73)90172-8.