Ensure that you have the RBAC plugin installed on the system where you are trying to add the group. This plugin should be on the installed list in the Installed tab of Manage Jenkins> Manage Plugins. If not then you need to manually install it in the Available tab.
Ensure that your external AD/LDAP groups are being parsed correctly by clicking on your user name in the top right:
Ensure that your RBAC group is not named the same name as the AD/LDAP group. This can cause name parsing issues and confusion
Ensure that the names of the external group and internal group are the same and have the same case since both AD and LDAP are case sensitive. This case sensitivity can be turned off on LDAP in the Manage Jenkins> Configure system page under the Case Sensitivity advanced tab: