R-devel removal of `R_ext/Callbacks.h`

[JanMarvin]

Dear all,

when I submitted a new release to CRAN earlier today, it was rejected due to a recent removal of R_ext/Callbacks.h in R devel: r-devel/r-svn@671453f

Rcpp/inst/include/RcppCommon.h

Line 76 in 2b0a44b

#include <R_ext/Callbacks.h>

I haven't had time to look into this further (e.g. if the Callback.h line can be removed). Just wanted to let you know. I have no rush and can wait, but presumably this might block all incoming packages with Rcpp as dependency.

[Enchufa2]

Added a long time ago:

Rcpp/ChangeLog

Lines 10871 to 10872 in 2b0a44b

	* src/RcppCommon.h: now includes <R_ext/Callbacks.h>, needed for the
	environment support

I don't think that's true anymore, and I don't see anything using anything from in there. Preparing PR.

[eddelbuettel]

@JanMarvin Thank you for the heads-up! I saw the email by R-Core member Luke Tierney and felt 'how nice that for once they give pro-active guidance' and did not even bother checking .... even though the same process of 'do not pre-announce, just remove' had just ambushed another package of mine (requiring new upload) and required a commit yesterday.

@Enchufa2 is already on it (wonder of timezones and 'near global dominance of Team Rcpp 😜 ' ...

[tylermorganwall]

Just encountered this myself as well on updating a package as part of CRAN's CXX11 cleanup: thanks for opening the issue, @JanMarvin.

[eddelbuettel]

@tylermorganwall This may be two very different kettle of fish.

CRAN has been at the 'please remove C++11 requirements unless truly needed' since the summer, and I quite appreciate their patient work there (which coincided with RcppArmadillo also moving on to C++14 but being kind and allowing a fallback because the scope was rather vast with more than 300+ out of 1200+ packages affected). An R Core commit to r-devel broke CRAN last night by breaking compilation under r-devel. They both look like compilation errors but they are miles apart. R-devel breaks every now and then at CRAN because it is effectively a 'larger scale CI system' for R. We will get it sorted out. A new Rcpp is slated for January after the CRAN reopen anyway.

[mikldk]

Thanks for acting so quickly. I, too, have this issue.

[tylermorganwall]

@tylermorganwall This may be two very different kettle of fish.

CRAN has been at the 'please remove C++11 requirements unless truly needed' since the summer, and I quite appreciate their patient work there (which coincided with RcppArmadillo also moving on to C++14 but being kind and allowing a fallback because the scope was rather vast with more than 300+ out of 1200+ packages affected). An R Core commit to r-devel broke CRAN last night by breaking compilation under r-devel. They both look like compilation errors but they are miles apart. R-devel breaks every now and then at CRAN because it is effectively a 'larger scale CI system' for R. We will get it sorted out. A new Rcpp is slated for January after the CRAN reopen anyway.

Yes, this is unrelated to the CXX11 update: I mentioned it because there may be a significant number of packages (about 60) encountering this right now, as CRAN recently (Nov 27th) sent out the notification to fix the C++11 req or be removed by 2025-12-18.

Here is a list:

db = tools::CRAN_package_db()

is_cpp11 = !is.na(db$SystemRequirements) &
  grepl("\\bC\\+\\+11\\b", db$SystemRequirements)

imports_rcpp = !is.na(db$Imports) &
  grepl("\\bRcpp\\b", db$Imports) &
  !is.na(db$Deadline)

# Count of packages meeting both conditions
db[is_cpp11 &imports_rcpp & !is.na(db$Deadline),c("Package", "Deadline")  ] |> print()
#>                   Package   Deadline
#> 128              adaHuber 2025-12-18
#> 324                 ahMLE 2025-12-18
#> 1326                  BCT 2025-12-18
#> 2322               CASMAP 2025-12-18
#> 3427            contoureR 2025-12-18
#> 3635             covTestR 2025-12-18
#> 3953                cusum 2025-12-18
#> 4270              DDRTree 2025-12-18
#> 4340            deformula 2025-12-18
#> 4674           disclapmix 2025-12-18
#> 4687          discretefit 2025-12-18
#> 5049               dsmisc 2025-12-18
#> 5081                  DtD 2025-12-18
#> 5168              dynatop 2025-12-18
#> 5514             elfDistr 2025-12-18
#> 5839                  ess 2025-12-18
#> 6210             FarmTest 2025-12-18
#> 6231              fastcmh 2025-12-18
#> 6257         fastkmedoids 2025-12-18
#> 6559                  FIT 2025-12-18
#> 6963           FSInteract 2025-12-18
#> 7219                  gbp 2025-12-18
#> 7646     gghilbertstrings 2025-12-18
#> 8260             gridtext 2025-12-18
#> 8367               gsynth 2025-12-18
#> 8817          HMMmlselect 2025-12-18
#> 8916                 HTLR 2025-12-18
#> 9136              ICRanks 2025-12-18
#> 9326              imptree 2025-12-18
#> 9791              jaccard 2025-12-18
#> 10136       knn.covertree 2025-12-18
#> 10355           lazyarray 2025-12-18
#> 11178           markerpen 2025-12-18
#> 12363              motmot 2025-12-18
#> 13930         outbreaker2 2025-12-18
#> 13975                 PAC 2025-12-18
#> 14137              parglm 2026-01-15
#> 16427  RcppGreedySetCover 2025-12-18
#> 17315              rminqa 2025-12-18
#> 17956                 rtk 2025-12-18
#> 18072             Rwofost 2025-12-18
#> 18652              semver 2025-12-18
#> 18780                 SFS 2025-12-18
#> 19169         simplextree 2026-01-15
#> 19403          SMITIDvisu 2025-12-18
#> 19647           sparkwarc 2025-12-18
#> 19664     sparseHessianFD 2025-12-18
#> 19721          SpatialKWD 2025-12-18
#> 20247 StepSignalMargiLike 2025-12-18
#> 20277             StMoSim 2025-12-18
#> 20364             strider 2025-12-18
#> 20871            TDAstats 2025-12-18
#> 20963        terrainmeshr 2025-12-18
#> 21059           TFMPvalue 2025-12-18
#> 21396        tokenbrowser 2025-12-18
#> 21546          TransPhylo 2025-12-18
#> 21686          trustOptim 2025-12-18
#> 21985               unine 2025-12-18

[eddelbuettel]

I am aware. Trust me that I have been following this rather closely on a daily basis given that I also sent out over sixty PRs or patches (covering all packages with at least one reverse dependency). Anyway, that question still has nothing to do with this issue so I stop here.

[JanMarvin]

Thank you for acting so swiftly @Enchufa2 ! Meanwhile R-Core had a change of heart and restored R_ext/Callbacks.h and R_ext/PrtUtil.h r-devel/r-svn@679f14e

[Enchufa2]

Great, so everyone can resubmit, to fix their packages before the deadline, and we can keep the release schedule without rushing to send a hot fix. :)

[eddelbuettel]

In the RSS feed summarizing NEWS I now see

• The non-API header files ‘R_ext/Callbacks.h’ and ‘R_ext/PrtUtil.h’ are no longer copied to the installed includes directory. The ‘R_ObjectTable’ type definitions formally in ‘R_ext/Callbacks.h’ are now available in the new header file ‘R_ext/ObjectTable.h’. This new header file should be used in packages instead of ‘R_ext/Callbacks.h’. To ease the transition a stub file ‘R_ext/Callbacks.h’ has been added that just includes ‘R_ext/ObjectTable.h’.

so maybe we need to add R_ext/ObjectTable.h? Or, as we don't seem to use the definition, maybe we don't 😁 But nice to see a more measured transition taking place upstream.

PS Motivation in that header:

/*
  The following definitions are for callbacks to R functions and
  methods related to user-level tables.  This was implemented in a
  separate package formerly available from Omegahat and these
  declarations allow the package to interface to the internal R code.
  
  See https://developer.r-project.org/RObjectTables.pdf.
*/

We do not need that for us. But I love that someone remembered that "exposing an API is a contract" and that your grandmother always reminded you to not break contracts for no apparent reason.

[eddelbuettel]

Everybody ok if we are closing this as R Core / Luke seem to have realized the previous approach that caused this issue was a no-go and reverted?

[eddelbuettel]

Thanks!

[eddelbuettel]

(CRAN is still a mess though. Running ciw::ciws() looking just at pretest lists 79 packages. Their processing is blocked / choked as we see with test error for knitr/rmarkdown missing or, of course 'Rcpp not installed'. It'll sort itself out in the next day or two.)