Your question itself is not naive but the type of answer you ask for is. It is rare for any line of work or intellectual enquiry to have an elevator pitch explanation. Not all would agree with your characterizations of mathematics and physics because they ignore the depth and nuances of those fields.
Theoretical computer scientists are concerned with studying and applying computation. The computational perspective is a deep and all encompassing one so the study of computation is also deep and has a bearing on many other areas of study. Every single process, whether arising in nature or synthetic, manipulates information. They compute. As in mathematics, there are different languages and types of structures involved in computation, as in physics, there are fundamental laws about computation that we are trying to discover, as in chemistry, fundamental elements of computation can be classified. Theoretical computer science is broad and robust enough to be amenable to any perspective you bring to it. Some of the questions studied are:
- What is computation and how can it be characterized? (Turing machines, lambda calculi, tiling systems, register machines, DNA computers etc.)
- What is the computational model underlying a process? (Biological, chemical, economic, sociological processes, etc.)
- What is efficient computation? (complexity via time, space, communication, amortized, smoothed, etc.)
- What are characterizations of efficient computation? (Turing machines, algebraic notions, logic, type systems, etc.)
- What is the most efficient way to compute a solution to a problem? (algorithms)
- How efficient are existing algorithmic processes (analysis of algorithms, statistical phenomena, market equillibria, etc.)
These are a small and non-representative sample of the questions that one may ask. As in any field, answers to some questions generate new questions and drive enquiry about other questions. You can find a rather dated view of the field by browsing the articles in the Handbook of Theoretical Computer Science.
- Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity, 1990
- Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics, 1990