-ldflags="-s -w" gets rid of the symbol table ("-s") for debugging purposes, and "-w" gets rid of the DWARF debugging information.
That's why it reduces size, and since it is not really needed to debug the code in production, this made sense for a good practice.
Example that was included in Dockerfile:
RUN CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -o server -a -ldflags="-s -w" -installsuffix cgo