To be clear that I understand the context, I'm going to first reproduce you situation worded differently.
Consider a sequence of sets, $C_i$, and a target function, $f$. Let $\cup C_i=C$. We wish to find a subset $S\subseteq C$ that maximizes $f(S)$ under the constraint that $\forall i, |S\cap C_i|<k_i$ and $|S|<k$.
I would agree with the commenter that this should be referred to as a cardinality constraint or a collection of cardinality constraints. I might also call it a "structural constraint," but I would specifically avoid calling it a "group carnality constraint" as "group" already has a meaning. In general, I would advocate for avoiding referring to a collection as a "group," "set," or "class" unless your object actually satisfies the required axioms. "Collection" is a good, general word that doesn't have a mathematical meaning.
If you don't want to use the term "carnality constraint" you can always introduce a term to refer to this specific restriction in your context. This can be helpful because it allows you to highlight the context-sensitive meaning of the constraint in a way that a more general term doesn't. For example, I have a paper with a constraint of this form that I refer to as an "isolation condition" because, when $S$ satisfies it, $S$ is isolated from the structure of the rest of the graph in a relevant way. This gives me a simple and intuitive way to refer to the condition, while simultaneously telling the reader what is important about the condition every time it comes up.