I would say that losing something is problematic in general.

[Because not having a hat whenever he went out was a problem, it was a very big problem.

... Maybe not a big problem in the grand scheme, but enough of one to put annoy him whenever he went to go out and turned instinctively to grab his hat.]

That said... I presume that it’s more about her reaction to having lost something that you mean to be concerned about.

[He remembered the tree, even if she had put it back together again it had been quite a... thing.]