setlevels(): avoid crash on missing factor values

NEWS.md

@@ -34,6 +34,8 @@
 
 7. Fixed compilation failure like "error: unknown type name 'siginfo_t'" in v1.18.0 in some strict environments, e.g., FreeBSD, where the header file declaring the POSIX function `waitid` does not transitively include the header file defining the `siginfo_t` type, [#7516](https://github.com/rdatatable/data.table/issues/7516). Thanks to @jszhao for the report and @aitap for the fix.
 
+8. When fixing duplicate factor levels, `setattr()` no longer crashes upon encountering missing factor values, [#7595](https://github.com/Rdatatable/data.table/issues/7595). Thanks to @sindribaldur for the report and @aitap for the fix.
+
 ### Notes
 
 1. {data.table} now depends on R 3.5.0 (2018).

inst/tests/tests.Rraw

@@ -22100,3 +22100,7 @@ test(2360.4, rowwiseDT(x =, expr =, 1, quote(a + b)),
   error = "Column 'expr' is type 'call'. Non-atomic, non-list objects must be wrapped in list\\(\\)")
 test(2360.5, rowwiseDT(x =, plist =, 1, as.pairlist(list(123))),
   error = "Column 'plist' is type 'pairlist'. Non-atomic, non-list objects must be wrapped in list\\(\\)")
+
+# setattr() must not crash for out-of-bounds factor indices when fixing duplicate levels, #7595
+test(2361.1, setattr(factor(c(1, NA), levels = 1), "levels", c("1", "1")), factor(c(1, NA)))
+test(2361.2, setattr(structure(c(-999L, 999L), class = "factor", levels = "a"), "levels", c("b", "b")), factor(c(NA, NA), levels = "b"))

src/wrappers.c

@@ -44,8 +44,11 @@ SEXP setlevels(SEXP x, SEXP levels, SEXP ulevels) {
   SEXP xchar, newx;
   xchar = PROTECT(allocVector(STRSXP, nx));
   int *ix = INTEGER(x);
-  for (int i=0; i<nx; ++i)
-    SET_STRING_ELT(xchar, i, STRING_ELT(levels, ix[i]-1));
+  const int nlevels = length(levels);
+  for (int i=0; i<nx; ++i) {
+    const int ixi = ix[i];
+    SET_STRING_ELT(xchar, i, (ixi >= 1 && ixi <= nlevels) ? STRING_ELT(levels, ix[i]-1) : NA_STRING);
+  }
   newx = PROTECT(chmatch(xchar, ulevels, NA_INTEGER));
   int *inewx = INTEGER(newx);
   for (int i=0; i<nx; ++i) ix[i] = inewx[i];