Fix Thread-Safety by Vipul-Cariappa · Pull Request #721 · compiler-research/CppInterOp

Vipul-Cariappa · 2025-09-24T07:47:24Z

This PR fixes the problems with the old implementation.
We now globally lock all the interpreters when we iterate over all the interpreters' bump allocators to resolve which interpreter a QualType belongs to. Similarly, we also lock all the interpreters to resolve the interpreter from a Decl.

I have also adopted most of the tests to run in parallel, hopefully detecting failures early.

Notes to myself:

You may want to skip InterpreterTest.MultipleInterpreters for LLVM 18. I have seen a crash in CI.
- No, I got that crash with LLVM 21 locally too. Might happen ~1/120 times.

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 149. Check the log or trigger a new build to see more.

github-actions · 2025-09-24T08:11:40Z

lib/CppInterOp/CppInterOp.cpp

+      if (IsSomeThreadWriting == 0)
+        return;
+      IsSomeThreadWritingMutex.unlock();
+      std::this_thread::sleep_for(0ms); // give control to other threads


warning: no header providing "std::operator""ms" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:41:

- #if CLANG_VERSION_MAJOR >= 19 + #include <bits/chrono.h> + #if CLANG_VERSION_MAJOR >= 19

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

+      if (IsSomeThreadWriting == 0)
+        return;
+      IsSomeThreadWritingMutex.unlock();
+      std::this_thread::sleep_for(0ms); // give control to other threads


warning: no header providing "std::this_thread::sleep_for" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:41:

- #if CLANG_VERSION_MAJOR >= 19 + #include <thread> + #if CLANG_VERSION_MAJOR >= 19

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

+//   return getSema(D).getASTContext();
+// }
+
+int Declare(compat::Interpreter& I, const char* code, bool silent);


warning: function 'Declare' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

int Declare(compat::Interpreter& I, const char* code, bool silent);

static int Declare(compat::Interpreter& I, const char* code, bool silent);

Cannot do this, Ref:

CppInterOp/lib/CppInterOp/CXCppInterOp.cpp

Line 327 in 0aa859b

int Declare(compat::Interpreter& interp, const char* code, bool silent);

Why Declare there is different than for example process:

CppInterOp/lib/CppInterOp/CXCppInterOp.cpp

Line 342 in 0aa859b

if (getInterpreter(I)->process(code))

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

 }

-static void InstantiateFunctionDefinition(Decl* D) {
+static void InstantiateFunctionDefinition(Decl* D, Interpreter& I) {


warning: no header providing "Cpp::Interpreter" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:12:

+ #include "CppInterOpInterpreter.h"

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

  auto* Within = llvm::dyn_cast<clang::DeclContext>(D);

-  auto* ND = Cpp_utils::Lookup::Named(&getSema(D), name, Within);
+  auto* ND = Cpp_utils::Lookup::Named(&I.getSema(), name, Within);


warning: no header providing "Cpp::utils::Lookup::Named" is directly included [misc-include-cleaner]

auto* ND = Cpp_utils::Lookup::Named(&I.getSema(), name, Within); ^

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

+}
+
+TCppIndex_t GetNumBases(TCppScope_t klass) {
+  auto* D = (Decl*)klass;


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

auto* D = (Decl*)klass; ^

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp


+TCppType_t GetTypeFromScope(TCppScope_t klass) {
+  if (!klass)
+    return 0;


warning: use nullptr [modernize-use-nullptr]

Suggested change

return 0;

return nullptr;

github-actions · 2025-09-24T08:11:41Z

lib/CppInterOp/CppInterOp.cpp

+  if (!klass)
+    return 0;
+
+  auto* D = (Decl*)klass;


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

auto* D = (Decl*)klass; ^

github-actions · 2025-09-24T08:11:42Z

lib/CppInterOp/CppInterOp.cpp

 // encompassed in an anonymous namespace as follows.
 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+static std::atomic<unsigned long long> gWrapperSerial = 0LL;


warning: 'gWrapperSerial' is a static definition in anonymous namespace; static is redundant here [readability-static-definition-in-anonymous-namespace]

Suggested change

static std::atomic<unsigned long long> gWrapperSerial = 0LL;

std::atomic<unsigned long long> gWrapperSerial = 0LL;

github-actions · 2025-09-24T08:11:42Z

lib/CppInterOp/CppInterOp.cpp

 // encompassed in an anonymous namespace as follows.
 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+static std::atomic<unsigned long long> gWrapperSerial = 0LL;


warning: no header providing "std::atomic" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:41:

- #if CLANG_VERSION_MAJOR >= 19 + #include <atomic> + #if CLANG_VERSION_MAJOR >= 19

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 139. Check the log or trigger a new build to see more.

github-actions · 2025-09-24T09:15:55Z

lib/CppInterOp/CppInterOp.cpp

 // encompassed in an anonymous namespace as follows.
 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+static std::atomic<unsigned long long> gWrapperSerial = 0LL;


warning: variable 'gWrapperSerial' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

static std::atomic<unsigned long long> gWrapperSerial = 0LL; ^

Shouldn't that be per-interpreter state?

Yes, ideally it should be. But we cannot make it per-interpreter if we want to support cling. At least it is complicated. I say will leave as a static atomic.

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOp.cpp

+    std::lock_guard<std::mutex> Lock(gWrapperStoreMutex);
+    auto R = gWrapperStore.find(FD);
+    if (R != gWrapperStore.end())
+      return (JitCall::GenericCall)R->second;


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

return (JitCall::GenericCall)R->second; ^

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOp.cpp

+    std::lock_guard<std::mutex> Lock(gDtorWrapperStoreMutex);
+    auto I = gDtorWrapperStore.find(D);
+    if (I != gDtorWrapperStore.end())
+      return (JitCall::DestructorCall)I->second;


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

return (JitCall::DestructorCall)I->second; ^

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOp.cpp

-  if (!Cpp::Declare(instance.c_str(), /*silent=*/false, interp)) {
-    auto* VD = static_cast<VarDecl*>(Cpp::GetNamed(id, nullptr, interp));
+  if (!Cpp::Declare(*IF, instance.c_str(), /*silent=*/false)) {
+    auto* VD = static_cast<VarDecl*>(Cpp::GetNamed(*IF, id, nullptr));


warning: do not use static_cast to downcast from a base to a derived class; use dynamic_cast instead [cppcoreguidelines-pro-type-static-cast-downcast]

Suggested change

auto* VD = static_cast<VarDecl*>(Cpp::GetNamed(*IF, id, nullptr));

auto* VD = dynamic_cast<VarDecl*>(Cpp::GetNamed(*IF, id, nullptr));

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOpInterpreter.h

-    llvm::InitializeAllTargets();
-    llvm::InitializeAllTargetMCs();
-    llvm::InitializeAllAsmPrinters();
+    static std::once_flag call_once_flag;


warning: no header providing "std::once_flag" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOpInterpreter.h:25:

- #if CLANG_VERSION_MAJOR >= 19 + #include <mutex> + #if CLANG_VERSION_MAJOR >= 19

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOpInterpreter.h

-    llvm::InitializeAllTargetMCs();
-    llvm::InitializeAllAsmPrinters();
+    static std::once_flag call_once_flag;
+    std::call_once(call_once_flag, []() {


warning: no header providing "std::call_once" is directly included [misc-include-cleaner]

std::call_once(call_once_flag, []() { ^

github-actions · 2025-09-24T09:15:56Z

lib/CppInterOp/CppInterOpInterpreter.h

  const clang::CompilerInstance* getCI() const { return getCompilerInstance(); }

  clang::Sema& getSema() const { return getCI()->getSema(); }
+  clang::ASTContext& getASTContext() const { return getSema().getASTContext(); }


warning: function 'getASTContext' should be marked [[nodiscard]] [modernize-use-nodiscard]

Suggested change

clang::ASTContext& getASTContext() const { return getSema().getASTContext(); }

[[nodiscard]] clang::ASTContext& getASTContext() const { return getSema().getASTContext(); }

github-actions · 2025-09-24T09:15:57Z

unittests/CppInterOp/EnumReflectionTest.cpp

 #include "gtest/gtest.h"

+#include <chrono>
+#include <thread>


warning: included header chrono is not used directly [misc-include-cleaner]

Suggested change

#include <thread>

#include <thread>

github-actions · 2025-09-24T09:15:57Z

unittests/CppInterOp/EnumReflectionTest.cpp


+#include <chrono>
+#include <thread>
+


warning: included header thread is not used directly [misc-include-cleaner]

Suggested change

github-actions · 2025-09-24T09:15:57Z

unittests/CppInterOp/EnumReflectionTest.cpp

 using namespace clang;

-TEST(ScopeReflectionTest, IsEnumScope) {
+void ScopeReflectionTest_IsEnumScope() {


warning: function 'ScopeReflectionTest_IsEnumScope' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void ScopeReflectionTest_IsEnumScope() {

static void ScopeReflectionTest_IsEnumScope() {

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 132. Check the log or trigger a new build to see more.

github-actions · 2025-09-24T11:35:17Z

lib/CppInterOp/CppInterOp.cpp

+#include <atomic>
 #include <cassert>
+#include <chrono>
 #include <cstddef>


warning: included header chrono is not used directly [misc-include-cleaner]

Suggested change

#include <cstddef>

#include <cstddef>

github-actions · 2025-09-24T11:35:18Z

lib/CppInterOp/CppInterOp.cpp

+      if (IsSomeThreadWriting == 0)
+        return;
+      IsSomeThreadWritingMutex.unlock();
+      std::this_thread::sleep_for(0ms); // give control to other threads


warning: no header providing "std::operator""ms" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:42:

- #if CLANG_VERSION_MAJOR >= 19 + #include <bits/chrono.h> + #if CLANG_VERSION_MAJOR >= 19

github-actions · 2025-09-24T11:35:18Z

lib/CppInterOp/CppInterOp.cpp

 // encompassed in an anonymous namespace as follows.
 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+std::atomic<unsigned long long> gWrapperSerial = 0LL;


warning: variable 'gWrapperSerial' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

std::atomic<unsigned long long> gWrapperSerial = 0LL; ^

github-actions · 2025-09-24T11:35:18Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(ScopeReflectionTest, IsEnumConstant) {
+void ScopeReflectionTest_IsEnumConstant() {


warning: function 'ScopeReflectionTest_IsEnumConstant' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void ScopeReflectionTest_IsEnumConstant() {

static void ScopeReflectionTest_IsEnumConstant() {

github-actions · 2025-09-24T11:35:18Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, IsEnumType) {
+void EnumReflectionTest_IsEnumType() {


warning: function 'EnumReflectionTest_IsEnumType' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_IsEnumType() {

static void EnumReflectionTest_IsEnumType() {

github-actions · 2025-09-24T11:35:18Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, GetIntegerTypeFromEnumScope) {
+void EnumReflectionTest_GetIntegerTypeFromEnumScope() {


warning: function 'EnumReflectionTest_GetIntegerTypeFromEnumScope' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_GetIntegerTypeFromEnumScope() {

static void EnumReflectionTest_GetIntegerTypeFromEnumScope() {

github-actions · 2025-09-24T11:35:19Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, GetIntegerTypeFromEnumType) {
+void EnumReflectionTest_GetIntegerTypeFromEnumType() {


warning: function 'EnumReflectionTest_GetIntegerTypeFromEnumType' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_GetIntegerTypeFromEnumType() {

static void EnumReflectionTest_GetIntegerTypeFromEnumType() {

github-actions · 2025-09-24T11:35:19Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, GetEnumConstants) {
+void EnumReflectionTest_GetEnumConstants() {


warning: function 'EnumReflectionTest_GetEnumConstants' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_GetEnumConstants() {

static void EnumReflectionTest_GetEnumConstants() {

github-actions · 2025-09-24T11:35:19Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, GetEnumConstantType) {
+void EnumReflectionTest_GetEnumConstantType() {


warning: function 'EnumReflectionTest_GetEnumConstantType' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_GetEnumConstantType() {

static void EnumReflectionTest_GetEnumConstantType() {

github-actions · 2025-09-24T11:35:19Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }

-TEST(EnumReflectionTest, GetEnumConstantValue) {
+void EnumReflectionTest_GetEnumConstantValue() {


warning: function 'EnumReflectionTest_GetEnumConstantValue' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

Suggested change

void EnumReflectionTest_GetEnumConstantValue() {

static void EnumReflectionTest_GetEnumConstantValue() {

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 30. Check the log or trigger a new build to see more.

github-actions · 2025-09-24T13:09:38Z

lib/CppInterOp/CppInterOp.cpp

  InterpreterInfo(const InterpreterInfo&) = delete;
  InterpreterInfo& operator=(const InterpreterInfo&) = delete;
+
+  clang::Sema& getSema() const { return Interpreter->getSema(); }


warning: function 'getSema' should be marked [[nodiscard]] [modernize-use-nodiscard]

Suggested change

clang::Sema& getSema() const { return Interpreter->getSema(); }

[[nodiscard]] clang::Sema& getSema() const { return Interpreter->getSema(); }

github-actions · 2025-09-24T13:09:38Z

lib/CppInterOp/CppInterOp.cpp

  InterpreterInfo& operator=(const InterpreterInfo&) = delete;
+
+  clang::Sema& getSema() const { return Interpreter->getSema(); }
+  clang::ASTContext& getASTContext() const {


warning: function 'getASTContext' should be marked [[nodiscard]] [modernize-use-nodiscard]

Suggested change

clang::ASTContext& getASTContext() const {

[[nodiscard]] clang::ASTContext& getASTContext() const {

github-actions · 2025-09-24T13:09:38Z

lib/CppInterOp/CppInterOp.cpp

  auto* Within = llvm::dyn_cast<clang::DeclContext>(D);

-  auto* ND = Cpp_utils::Lookup::Named(&getSema(D), name, Within);
+  auto* ND = Cpp_utils::Lookup::Named(&I.getSema(), name, Within);


warning: no header providing "Cpp::utils::Lookup::Named" is directly included [misc-include-cleaner]

lib/CppInterOp/CppInterOp.cpp:12:

+ #include "CppInterOpInterpreter.h"

github-actions · 2025-09-24T13:09:39Z

lib/CppInterOp/CppInterOp.cpp

 // encompassed in an anonymous namespace as follows.
 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+std::atomic<unsigned long long> gWrapperSerial =


warning: variable 'gWrapperSerial' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

std::atomic<unsigned long long> gWrapperSerial = ^

github-actions · 2025-09-24T13:09:39Z

lib/CppInterOp/CppInterOp.cpp

+    std::lock_guard<std::mutex> Lock(gWrapperStoreMutex);
+    auto R = gWrapperStore.find(FD);
+    if (R != gWrapperStore.end())
+      return (JitCall::GenericCall)


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

return (JitCall::GenericCall) ^

github-actions · 2025-09-24T13:09:39Z

lib/CppInterOp/CppInterOp.cpp

+    std::lock_guard<std::mutex> Lock(gDtorWrapperStoreMutex);
+    auto I = gDtorWrapperStore.find(D);
+    if (I != gDtorWrapperStore.end())
+      return (JitCall::DestructorCall)


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

return (JitCall::DestructorCall) ^

github-actions · 2025-09-24T13:09:39Z

unittests/CppInterOp/EnumReflectionTest.cpp

 }
+
+TEST(EnumReflectionTest, EnumReflectionTest) {
+  std::vector<std::pair<const char*, void (*)()>> fns = {


warning: no header providing "std::pair" is directly included [misc-include-cleaner]

unittests/CppInterOp/EnumReflectionTest.cpp:7:

+ #include <utility>

github-actions · 2025-09-24T13:09:39Z

unittests/CppInterOp/FunctionReflectionTest.cpp

 using namespace clang;

-TEST(FunctionReflectionTest, GetClassMethods) {
+static Cpp::TInterp_t I = nullptr;


warning: variable 'I' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

static Cpp::TInterp_t I = nullptr; ^

github-actions · 2025-09-24T13:09:39Z

unittests/CppInterOp/FunctionReflectionTest.cpp

    )";

-  GetAllTopLevelDecls(code, Decls);
+  Cpp::TInterp_t I1 = GetAllTopLevelDecls(code, Decls);


warning: Value stored to 'I1' during its initialization is never read [clang-analyzer-deadcode.DeadStores]

Cpp::TInterp_t I1 = GetAllTopLevelDecls(code, Decls); ^

Additional context

unittests/CppInterOp/FunctionReflectionTest.cpp:53: Value stored to 'I1' during its initialization is never read

Cpp::TInterp_t I1 = GetAllTopLevelDecls(code, Decls); ^

github-actions · 2025-09-24T13:09:39Z

unittests/CppInterOp/FunctionReflectionTest.cpp

+  EXPECT_FALSE(Cpp::Process(code.c_str(), I));
  const char* str = "std::make_unique<int,int>";
-  auto* Instance1 = (Decl*)Cpp::InstantiateTemplateFunctionFromString(str);
+  auto* Instance1 = (Decl*)Cpp::InstantiateTemplateFunctionFromString(str, I);


warning: do not use C-style cast to convert between unrelated types [cppcoreguidelines-pro-type-cstyle-cast]

auto* Instance1 = (Decl*)Cpp::InstantiateTemplateFunctionFromString(str, I); ^

github-actions

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 10 out of 12. Check the log or trigger a new build to see more.

github-actions · 2025-09-26T08:30:10Z

unittests/CppInterOp/FunctionReflectionTest.cpp

+  I = Cpp::CreateInterpreter({"-include", "new"});
+  EXPECT_TRUE(I);
+
+  std::vector<std::pair<const char*, void (*)()>> fns = {


warning: no header providing "std::pair" is directly included [misc-include-cleaner]

unittests/CppInterOp/FunctionReflectionTest.cpp:16:

+ #include <utility>

github-actions · 2025-09-26T08:30:10Z

unittests/CppInterOp/ScopeReflectionTest.cpp


-TEST(ScopeReflectionTest, Demangle) {
+// NOLINENEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
+static Cpp::TInterp_t I = nullptr;


warning: variable 'I' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

static Cpp::TInterp_t I = nullptr; ^

github-actions · 2025-09-26T08:30:10Z

unittests/CppInterOp/TypeReflectionTest.cpp


-TEST(TypeReflectionTest, GetTypeAsString) {
+// NOLINENEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
+static Cpp::TInterp_t I = nullptr;


warning: variable 'I' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

static Cpp::TInterp_t I = nullptr; ^

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/Utils.cpp


 #include <algorithm>
+#include <chrono>
+#include <future>


warning: included header chrono is not used directly [misc-include-cleaner]

Suggested change

#include <future>

#include <future>

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/Utils.cpp

    const std::vector<const char*>& interpreter_args /* = {} */) {
-  Cpp::CreateInterpreter(interpreter_args);
+  Cpp::TInterp_t I = Cpp::CreateInterpreter(interpreter_args);
+  auto* Interp = static_cast<compat::Interpreter*>(I);


warning: no header providing "compat::Interpreter" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:2:

- #include "CppInterOp/CppInterOp.h" + #include "/github/workspace/lib/CppInterOp/Compatibility.h" + #include "CppInterOp/CppInterOp.h"

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/Utils.cpp

+    std::vector<std::pair<const char*, void (*)()>>& fns, unsigned runners) {
+  std::vector<std::future<void>> futures;
+  std::vector<const char*> names;
+  for (size_t i = 0, size = fns.size(); i < size;) {


warning: no header providing "size_t" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:13:

- #include <future> + #include <cstddef> + #include <future>

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/Utils.cpp

+      i++;
+      continue;
+    }
+    assert(futures.size() == names.size());


warning: no header providing "assert" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:12:

- #include <chrono> + #include <cassert> + #include <chrono>

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/Utils.cpp

+    }
+    assert(futures.size() == names.size());
+    for (size_t i = 0, size = futures.size(); i < size; i++) {
+      if (futures[0].wait_for(0ms) == std::future_status::ready) {


warning: no header providing "std::operator""ms" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:12:

- #include <chrono> + #include <bits/chrono.h> + #include <chrono>

github-actions · 2025-09-26T08:30:11Z

unittests/CppInterOp/VariableReflectionTest.cpp


-TEST(VariableReflectionTest, GetDatamembers) {
+// NOLINENEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
+static Cpp::TInterp_t I = nullptr;


warning: variable 'I' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

static Cpp::TInterp_t I = nullptr; ^

github-actions · 2025-09-26T08:30:12Z

unittests/CppInterOp/VariableReflectionTest.cpp


-TEST(VariableReflectionTest, DatamembersWithAnonymousStructOrUnion) {
+    void
+    VariableReflectionTest_DatamembersWithAnonymousStructOrUnion() {


warning: function 'VariableReflectionTest_DatamembersWithAnonymousStructOrUnion' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

unittests/CppInterOp/VariableReflectionTest.cpp:119:

- void + static void

github-actions

clang-tidy made some suggestions

github-actions · 2025-09-26T09:15:49Z

unittests/CppInterOp/Utils.cpp

 #include <algorithm>
+#include <cassert>
+#include <chrono>
+#include <cstddef>


warning: included header chrono is not used directly [misc-include-cleaner]

Suggested change

#include <cstddef>

#include <cstddef>

github-actions · 2025-09-26T09:15:49Z

unittests/CppInterOp/Utils.cpp

+    }
+    assert(futures.size() == names.size());
+    for (size_t i = 0, size = futures.size(); i < size; i++) {
+      if (futures[0].wait_for(0ms) == std::future_status::ready) {


warning: no header providing "std::operator""ms" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:12:

- #include <cassert> + #include <bits/chrono.h> + #include <cassert>

github-actions · 2025-09-26T09:15:49Z

unittests/CppInterOp/VariableReflectionTest.cpp


-TEST(VariableReflectionTest, GetVariableOffset) {
+    void
+    VariableReflectionTest_GetVariableOffset() {


warning: function 'VariableReflectionTest_GetVariableOffset' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

unittests/CppInterOp/VariableReflectionTest.cpp:273:

- void + static void

github-actions · 2025-09-26T09:15:50Z

unittests/CppInterOp/VariableReflectionTest.cpp


-TEST(VariableReflectionTest, VariableOffsetsWithInheritance) {
+    void
+    VariableReflectionTest_VariableOffsetsWithInheritance() {


warning: function 'VariableReflectionTest_VariableOffsetsWithInheritance' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

unittests/CppInterOp/VariableReflectionTest.cpp:390:

- void + static void

github-actions

clang-tidy made some suggestions

github-actions · 2025-10-04T19:09:41Z

unittests/CppInterOp/Utils.cpp

+    }
+    assert(futures.size() == names.size());
+    for (size_t i = 0, size = futures.size(); i < size; i++) {
+      if (futures[0].wait_for(0ms) == std::future_status::ready) {


warning: no header providing "std::operator""ms" is directly included [misc-include-cleaner]

unittests/CppInterOp/Utils.cpp:14:

- #include <cassert> + #include <bits/chrono.h> + #include <cassert>

vgvassilev · 2025-10-05T16:09:23Z

lib/CppInterOp/CppInterOp.cpp

    sInterpreterASTMap;
-static std::recursive_mutex InterpreterStackLock;
-static std::recursive_mutex LLVMLock;
+static int IsSomeThreadWriting = 0;


We need probably a better name here IsThreadWriting is not too bad.

vgvassilev · 2025-10-05T16:09:59Z

lib/CppInterOp/CppInterOp.cpp

+static int IsSomeThreadWriting = 0;
+static std::mutex IsSomeThreadWritingMutex;
+static std::mutex InterpreterStackLock;
+static std::mutex LLVMLock;


Please add a comment explaining why these variables is fine to be process-wide globals.

vgvassilev · 2025-10-05T16:10:12Z

lib/CppInterOp/CppInterOp.cpp

+static std::mutex LLVMLock;
 // NOLINTEND(cppcoreguidelines-avoid-non-const-global-variables)

+class LOCK {


Why all caps?

vgvassilev · 2025-10-05T16:10:35Z

lib/CppInterOp/CppInterOp.cpp

+  LOCK operator=(LOCK&&) = delete;
+};
+
+struct WAIT_ON_ALL {


All caps again, we need doxygen style documentation.

vgvassilev · 2025-10-05T16:11:49Z

lib/CppInterOp/CppInterOp.cpp

+  //   getInterpInfo(result); // getASTContext may read wrong data
+  //    if definition is being written,
+  //    and the declaration has already been written
+  WAIT_ON_ALL WAIT;


We should name that as somethingRaii.

vgvassilev · 2025-10-05T16:17:40Z

lib/CppInterOp/CppInterOp.cpp

+  auto& IF = getInterpInfo(D);
+  auto& C = IF.getASTContext();


vgvassilev · 2025-10-05T16:19:27Z

lib/CppInterOp/CppInterOp.cpp

-TCppIndex_t GetNumBases(TCppScope_t klass) {
-  auto* D = (Decl*)klass;
-
+static TCppIndex_t GetNumBases(compat::Interpreter& I,


Why we have these clones of their counterparts taking I. I mean I know why but why only a handful of them and not all?

vgvassilev · 2025-10-05T16:21:27Z

lib/CppInterOp/CppInterOp.cpp

 namespace {
-static unsigned long long gWrapperSerial = 0LL;
+// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
+std::atomic<unsigned long long> gWrapperSerial = 0LL;


Should that not be unsigned long long?

vgvassilev · 2025-10-05T16:23:11Z

unittests/CppInterOp/EnumReflectionTest.cpp

 using namespace clang;

-TEST(ScopeReflectionTest, IsEnumScope) {
+static void EnumReflectionTest_IsEnumScope() {


These changes are incorrect. We should ignore clang-tidy here.

vgvassilev · 2025-10-05T16:24:33Z

unittests/CppInterOp/EnumReflectionTest.cpp

+  Cpp::TCppScope_t globalscope = Cpp::GetScope("", nullptr, I);
+  Cpp::TCppScope_t Animals_scope = Cpp::GetScope("Animals", nullptr, I);
+  Cpp::TCppScope_t myClass_scope = Cpp::GetScope("myClass", nullptr, I);
+  Cpp::TCppScope_t unsupported_scope = Cpp::GetScope("myVariable", nullptr, I);


I'd think the interfaces became very clunky here. We need something better...

Having static locals can be risky and we might want to move them into the InterpreterInfo class

We now globally lock all the interpreters when we iterate over all the interpreters' bump allocators to resolve which interpreter a `QualType` belongs to. Similarly, we also lock all the interpreters to resolve the interpreter from a `Decl`. I have also adopted most of the tests to run in parallel, hopefully detecting failures early.

Vipul-Cariappa · 2025-10-07T14:57:18Z

include/CppInterOp/CppInterOp.h

 /// matter, since the library will function in the same way.
 ///\returns the current interpreter instance, if any.
-CPPINTEROP_API TInterp_t GetInterpreter();
+CPPINTEROP_API TInterp_t GetInterpreter(TInterp_t I = nullptr);


Return value will be the same as argument. This should not be done.

Vipul-Cariappa requested review from aaronj0, anutosh491, mcbarton and vgvassilev September 24, 2025 07:47

github-actions bot reviewed Sep 24, 2025

View reviewed changes

github-actions bot reviewed Sep 26, 2025

View reviewed changes

Vipul-Cariappa force-pushed the wip/Thread branch 2 times, most recently from 54695d4 to 501440b Compare September 26, 2025 09:15

github-actions bot reviewed Sep 26, 2025

View reviewed changes

Vipul-Cariappa force-pushed the wip/Thread branch 2 times, most recently from 3d262b2 to 1fb2714 Compare September 26, 2025 10:23

Vipul-Cariappa mentioned this pull request Oct 2, 2025

Out-Of-Process Interpreter for CppInterOp #717

Merged

Vipul-Cariappa force-pushed the wip/Thread branch from 1fb2714 to ec0fd22 Compare October 4, 2025 18:50

github-actions bot reviewed Oct 4, 2025

View reviewed changes

vgvassilev reviewed Oct 5, 2025

View reviewed changes

Vipul-Cariappa added 2 commits October 6, 2025 16:19

fix thread safety for static locals

65c3587

Having static locals can be risky and we might want to move them into the InterpreterInfo class

mcbarton force-pushed the wip/Thread branch from ec0fd22 to f690e2d Compare October 6, 2025 15:19

Vipul-Cariappa commented Oct 7, 2025

View reviewed changes

	int Declare(compat::Interpreter& I, const char* code, bool silent);
	static int Declare(compat::Interpreter& I, const char* code, bool silent);

	static std::atomic<unsigned long long> gWrapperSerial = 0LL;
	std::atomic<unsigned long long> gWrapperSerial = 0LL;

	auto* VD = static_cast<VarDecl>(Cpp::GetNamed(IF, id, nullptr));
	auto* VD = dynamic_cast<VarDecl>(Cpp::GetNamed(IF, id, nullptr));

	clang::ASTContext& getASTContext() const { return getSema().getASTContext(); }
	[[nodiscard]] clang::ASTContext& getASTContext() const { return getSema().getASTContext(); }

	void ScopeReflectionTest_IsEnumScope() {
	static void ScopeReflectionTest_IsEnumScope() {

	void ScopeReflectionTest_IsEnumConstant() {
	static void ScopeReflectionTest_IsEnumConstant() {

	void EnumReflectionTest_IsEnumType() {
	static void EnumReflectionTest_IsEnumType() {

	void EnumReflectionTest_GetIntegerTypeFromEnumScope() {
	static void EnumReflectionTest_GetIntegerTypeFromEnumScope() {

	void EnumReflectionTest_GetEnumConstants() {
	static void EnumReflectionTest_GetEnumConstants() {

	void EnumReflectionTest_GetEnumConstantType() {
	static void EnumReflectionTest_GetEnumConstantType() {

	void EnumReflectionTest_GetEnumConstantValue() {
	static void EnumReflectionTest_GetEnumConstantValue() {

	clang::Sema& getSema() const { return Interpreter->getSema(); }
	[[nodiscard]] clang::Sema& getSema() const { return Interpreter->getSema(); }

Conversation

Vipul-Cariappa commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Vipul-Cariappa Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

vgvassilev Oct 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

vgvassilev Sep 27, 2025

Choose a reason for hiding this comment

Uh oh!

Vipul-Cariappa Oct 4, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Sep 24, 2025

Vipul-Cariappa commented Sep 24, 2025 •

edited

Loading