I was doing it in GF(2^32-5). Your statement is true for GF(2^n) where n is small enough to keep the entire multiplication-table in memory (usually n <= 8). When it's bigger you keep log-tables in memory then sign matters. However when n=16 you get lucky and can use char as an unsigned 16 bit int.
Addition is XOR which is sign-agnostic. Multiplication has to be done via table lookups to be fast which also makes it sign agnostic.
Well, at least for p=2.