The formation of elemental sulfur requires H2S, oxygen and liquid water. For elemental sulfur to form in the spent air stream there would have to be H2S that was not contacted by NaOH in the prewash caustic or in the extractor. Maybe the caustic is becoming too far spent before being recharged and that is when the H2S is breaking through.
A thorough review of the LPG caustic treater operation may reduce the problems you are experiencing. If not, then consider installing a larger knock out pot on the spent air line to prevent the liquid carry over you mentioned from getting into the incinerator.