[SalesForce] Fire custom event inside timeout

No. Because of CSS Isolation, you can't generally monkey with the internals of a component, especially if it's in another namespace. The documentation says:

CSS styles defined in a parent component don’t leak into a child. In our example, a p style defined in the todoApp.css style sheet doesn’t style the p element in the c-todo-item component, because the styles don’t reach into the shadow tree. See CSS.

There is currently a loadStyle hack that works around this, but I expect it will eventually be fixed.

You should be able to specify a height for the entire table, which should auto-size the inner table accordingly:

lightning-datatable {
  min-height: 10rem;
}

This is because the Shadow DOM allows us to modify the "outside" of the component, such as its size, a border, etc, while not allowing us to modify the internal components.

Your event name does not match here.

<c:customlwc onhandleResponse="{!c.handleResponse}"/>

You are firing event with name = handleSCResponse, so you need to prefix on to your event name, ex onhandlescresponse. The corrected code of handling it from the parent component is below.

<c:customlwc onhandlescresponse="{!c.handleResponse}"/>

So in detail, this is how you fire an event.

const eventName = 'onhandlescresponse';
const event = new CustomEvent(eventName, {
    detail: { message: 'Operation completed successfully' }
});
this.dispatchEvent(event);

This is how you handle it.

<aura:component implements="force:appHostable,flexipage:availableForAllPageTypes,flexipage:availableForRecordHome,force:hasRecordId,forceCommunity:availableForAllPageTypes,force:lightningQuickAction">
    <c:customlwc onhandlescresponse="{!c.handleResponse}"/>
</aura:component>

Although it is supported in the aura parent, its recommended to give the lowercase name for an event.