I haven't figured this out completely yet, hard to find the info and hard to interpret the info (for my limited brain at least), but let's go from this:

Error Information (NVMe Log 0x01, 16 of 256 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc          LBA  NSID    VS
  0        129     0  0xa013  0x8004  0x000            0     0     -

I am at this point interested in the status code. I found this: https://github.com/linux-nvme/nvme-cli/issues/800 and I am looking at answer from "birkelund" (NVMe Software Engineer), someone is asking for status code 0xc502 and he explains you decode like so:

If you are asking how that error code is encoded in 0xC502, then its 0xC502 >> 1 to get rid of the Phase Tag. That leave us with 0x6281. Then apply a mask of 0x7ff to extract the lower 11 bytes (3 for the Status Code Type and 8 for the Status Code), ending up with 0x281. 0x2xx are "Media and Data Integrity Errors" and the 0x81 status code is "Unrecovered Read Error".

In more human language the bit shift is a division by 2 (0xC502 / 2 = 0x6281). Applying mask 0x7ff gives us the 3 right-side nibbles (0x6281 -> 0x281). I think this makes decoding a tad easier.

To see how he gets Status Code Type and Status Code I did some more research:

Lookup for code types:

NVME_SCT_GENERIC        = 0x0,
NVME_SCT_COMMAND_SPECIFIC   = 0x1,
NVME_SCT_MEDIA_ERROR        = 0x2,
/* 0x3-0x6 - reserved */
NVME_SCT_VENDOR_SPECIFIC    = 0x7,

Lookup for "Media and Data Integrity Errors":

NVME_SC_WRITE_FAULTS            = 0x80,
NVME_SC_UNRECOVERED_READ_ERROR      = 0x81,
NVME_SC_GUARD_CHECK_ERROR       = 0x82,
NVME_SC_APPLICATION_TAG_CHECK_ERROR = 0x83,
NVME_SC_REFERENCE_TAG_CHECK_ERROR   = 0x84,
NVME_SC_COMPARE_FAILURE         = 0x85,
NVME_SC_ACCESS_DENIED           = 0x86,

So we can now see where he gets:

0x2xx are "Media and Data Integrity Errors" and the 0x81 status code is "Unrecovered Read Error"

If I apply same logic/method to status 0x8004 I get something like:

Shift Right: Shifting the value 0x8004 by one bit to the right (0x8004 >> 1), gets 0x4002. Then the masking: Applying a mask of 0x7ff extracts the lower 11 bits of the value 0x4002, yielding 0x002.

So 0 gets us NVME_SCT_GENERIC Status Code Type (see above table) and Generic Status Codes lookup:

NVME_SC_SUCCESS             = 0x00,
NVME_SC_INVALID_OPCODE          = 0x01,
NVME_SC_INVALID_FIELD           = 0x02,
NVME_SC_COMMAND_ID_CONFLICT     = 0x03,
NVME_SC_DATA_TRANSFER_ERROR     = 0x04,
NVME_SC_ABORTED_POWER_LOSS      = 0x05,
NVME_SC_INTERNAL_DEVICE_ERROR       = 0x06,
NVME_SC_ABORTED_BY_REQUEST      = 0x07,
NVME_SC_ABORTED_SQ_DELETION     = 0x08,
NVME_SC_ABORTED_FAILED_FUSED        = 0x09,
NVME_SC_ABORTED_MISSING_FUSED       = 0x0a,
NVME_SC_INVALID_NAMESPACE_OR_FORMAT = 0x0b,
NVME_SC_COMMAND_SEQUENCE_ERROR      = 0x0c,

NVME_SC_LBA_OUT_OF_RANGE        = 0x80,
NVME_SC_CAPACITY_EXCEEDED       = 0x81,
NVME_SC_NAMESPACE_NOT_READY     = 0x82,

So, now we have type error 0 (NVME_SCT_GENERIC = 0x0) and 02 for status (NVME_SC_INVALID_FIELD = 0x02). I have no idea what it means, but to me it does not sound like an issue with your NVMe drive itself. If SMART is 'clean' I think you have not much to worry about.

Looking for a further explanation I found:

NVME_SC_INVALID_FIELD - Invalid Field in Command: A reserved coded value or an unsupported value in a defined field.

Also, as far as I can tell CmdId (0xa013 in this case) is not an actual command but an ID for a 'structure' or packet that contains an actual command and parameters that you can pass on to the queue. So in itself CmdId 0xa013 tells us nothing about the actual command the host was sending to the drive.

Disclaimer: Math, bit-shifting and all that is not my strong point so I may have made an error, a typo or whatever in the calculator, you should check it before relying on it.

I think it is highly likely this is caused by this bug in smartmontools. Someone participating in this thread wrote about the cause of the messages:

The NVMe specification is not very consistent on how to identify what features the controller supports, so in some cases the driver just has to try it and see if it worked.

The log entries are likely harmless driver initiated admin commands (SqId 0) checking if a particular feature is supported. The SSD doesn't need to log an error entry for such commands as it has no impact on media health (which is what SMART is supposed to care about), but it is allowed to save the error if it wants. I personally find these types of errors to be less than useless.

The bug in smartmontools, i.e. the useless messages, was solved here, and it is not present in the release 7.4 (changelog here – "smartd: No longer issues LOG_CRIT warnings if new entries of NVMe error information log do not indicate device problems."). However, the smartmontools bug report has a followup, and it is not clear to me whether it has been solved yet.

I had the same problem. In my case, S.M.A.R.T had been working properly on the device for years while using Ubuntu 12.04, and then under Ubuntu 14.04 it happened exactly what you tell in the question.

The problem is related to a new kernel module that was introduced in Linux Kernel 3.15 called uas (USB Attached SCSI) (see release announcement).

That module is now the responsible of managing USB Mass Storage Devices. There is a thread where people complain that uas in kernel 3.15 is causing their USB devices to fail. Another one says that it might be the cause of S.M.A.R.T problems.

Fortunately, those problems seem to be gone at kernel 3.19 (which I am using), as my device is being detected correctly. Only the S.M.A.R.T problem remains.

To fix it, you need to disable the use of uas module for the given device.

Disable uas without rebooting

First, unplug all USB devices that might be using it. Then, remove the uas and usb-storage modules:

sudo modprobe -r uas
sudo modprobe -r usb-storage

Then, load usb-storage module with a parameter that tells it to not use uas for a given device:

sudo modprobe usb-storage quirks=VendorId:ProductId:u

VendorId and ProductId must be replaced by your device vendor and product id, which can be obtained with lsusb command (they are the characters after ID).

For example, I have the following device:

Bus 002 Device 011: ID 0bc2:3320 Seagate RSS LLC SRD00F2 [Expansion Desktop Drive]

So my vendor id is 0bc2, and my product id is 3320. My command is:

sudo modprobe usb-storage quirks=0bc2:3320:u

The last u tells usb-storage to ignore uas for the device (see source).

At this point, you can insert the USB device, and it will know not to use uas, making S.M.A.R.T work properly. You will see lines like these in dmesg when inserting the USB device:

usb 2-2: UAS is blacklisted for this device, using usb-storage instead
usb-storage 2-2:1.0: USB Mass Storage device detected
usb-storage 2-2:1.0: Quirks match for vid 0bc2 pid 3320: 800000
scsi host12: usb-storage 2-2:1.0

Make the change permanent

The previous quirk will only last until you reboot the system. To make it persistent, you need to follow the steps described here, which I copy below:

First, create a file named ignore_uas.conf in the /etc/modprobe.d/ directory with the following content:

options usb-storage quirks=VendorId:ProductId:u

As before, substitute VendorId and ProductId by your device vendor and product id obtained from lsusb.

Next, regenerate your inital ramdisk:

mkinitcpio -p linux

or, on newer Ubuntu versions:

sudo update-initramfs -u

Finally, reboot your computer.

Edit: More background on the issue, and another way to get around it without disabling uas (which has better throughput than usb-storage) can be found here: https://www.smartmontools.org/ticket/971#comment:12

It seems that kernel is blacklisting SAT ATA PASS-THROUGH on some devices when running in uas mode, as they have broken firmware.

So, the blacklisting can be disabled (at your own risk) by using the previous method I mention in the answer, but removing the final u from the quirk, ie:

quirks=VendorId:ProductId:

Please note, however, that I have not tested this approach.

The SMART status may not be available for the namespace (n1) or partition (p2). Hence you must call it for the device itself:

smartctl -x /dev/nvme0

You can override the namespace to be queried for with -d nvme,$nsid, and 0xffffffff is the "broadcast namespace id". By default smartctl selects $nsid from the device node namespace id (in your case 1). So to query with broadcast:

smartctl -x -d nvme,0xffffffff /dev/nvme0n1p2

Often you can ignore error log entries, it may for example be 'errors' due to the host sending non NVMe commands to the NVMe drive.

A sudden increase may be result of some (monitoring?) software for example you started using that sends queries to the NVMe drive. To be sure, the only way to find out what the error log entries are about is viewing what's inside them.

I don't know how this can be done using other tools. If you have a Windows system, you can try install HD Sentinel, select the NVMe drive ⇒ click Disk menu ⇒ Device Specific Information. You'll then be able to read the NVMe Error log.

Edit: See https://www.smartmontools.org/ticket/1300 for more suggestions to see the actual log entries:

sudo nvme error-log /dev/nvme0

(nvme-error-log(1) man page)