Q1. Below is full description for LOM mode:
Listen-Only Mode
This bit configures FlexCAN to operate in Listen-Only mode. In this mode, transmission is disabled, all error counters described in the ECR register are frozen, and the module operates in a CAN Error Passive mode. Only messages acknowledged by another CAN station will be received. If FlexCAN detects a message that has not been acknowledged, it will flag a BIT0 error without changing the receive error counter (ECR[RXERRCNT]), as if it was trying to acknowledge the message.
Listen-Only mode is acknowledged by the state of ESR1[FLTCONF] indicating Passive Error. There can be some delay between the Listen-Only mode request and acknowledge.
This bit can be written in Freeze mode only because it is blocked by hardware in other modes.
0b - Listen-Only mode is deactivated.
1b - FlexCAN module operates in Listen-Only mode.
In your setup, FlexCAN4 should be in LOM. That's why I said FlexCAN4 cannot work independently in this case, it can only receive messages that are also received by FlexCAN1. Then an incoming frame will be received by both FlexCAN1 and FlexCAN4 (meaning one MB per CAN controller => don't help extend MBs).
Q2. Yes, but note that this idea only works with my hypothesis - "Then you can just use FlexCAN1 for 11 Tx CAN messages, and use both FlexCAN1 and FlexCAN4 for 53 Rx CAN messages (e.g. FlexCAN1 will receive some CAN messages ID and FlexCAN4 will take care of the rest)". To be honest, I have no idea if your scenario is 53 Rx CAN messages with same message ID.
Q3. I'll try to clarify more on this thread to see if I'm missing something