Thoughts on C++ optimization (LP solving)

Hi @andrschl, @michalsustr, @gabrfarina,
I've recently been working on LP-solving from C++, using OR-Tools. I have a simple example that solves a matrix game (not yet released). It works like a charm internally, but I am having trouble externally. I managed to get it to compile and link against the binary distribution but it crashes when I run it, and in so doing I discovered the wonderful world of ABI incompatibilities between C++ standards, so that's not an option. When I try building from source I get a bunch of errors with cmake (see below: OR-Tools's CMake setup is marked as experimental, so this could just be something that is not supported yet).
I will include the code in the next update including my attempt to get it to work with CMake if you're curious. But, this has led me to consider other LP solvers. OR-Tools is very appealing because it checks off all the boxes: works internally and externally, and problems can be solved using some of the large commercial solvers. But it a lot of dependencies, so building from scratch could take some time and long-term could be hard to maintain. I've used GLPK in the past, and it worked quite well, but building the LP code on GLPK would mean it can't transfer seamlessly to e.g. CPLEX or Gurobi.
So I'm a bit torn.. I still want to try to get OR-Tools working, especially because they support a CMake build (!) which is great for us, but what I know for sure is that I can't spend much more time on this at the moment. I'm wondering what you have used in the past to do your LP solving, and was it from C++? Any recommendations on where to go from here?
Here's the promised CMake error message:
CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "open_spiel_core" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "container" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "flags" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "flags_parse" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "flat_hash_map" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "optional" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "random_random" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "str_format" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "strings" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/gflags-src/CMakeLists.txt:
  export called with target "gflags_nothreads_static" which requires target
  "time" that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "open_spiel_core"
  that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "container" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "flags" that is not
  in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "flags_parse" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "flat_hash_map" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "optional" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "random_random" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "str_format" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "strings" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/glog-src/CMakeLists.txt:
  export called with target "glog" which requires target "time" that is not
  in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "open_spiel_core"
  that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "container" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "flags" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "flags_parse"
  that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "flat_hash_map"
  that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "optional" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "random_random"
  that is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "str_format" that
  is not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "strings" that is
  not in any export set.

CMake Error in /usr/local/google/home/lanctot/tmp/open_spiel/build/_deps/scip-src/src/CMakeLists.txt:
  export called with target "libscip" which requires target "time" that is
  not in any export set.

When I try a different route to link ortools::ortools to open_spiel_core, I get a different error about strong connected cycles in the dependency graph. I appreciate that it's hard to see the problem without seeing the code, so I'll post a reminder when it's in. But I'd still like your thoughts on what you've used to solve your LPs.

I have used GLPK several times for linear programming, and it is orders of magnitude slower than Gurobi/CPLEX. I think it's important to support also commercial software.
Also, I don't know about it's MIP solver but I imagine it's nowhere close to commercial software. All implementations of algorithms for team correlation/collusion that I can think of relies on a mip solver.
Using a library that abstract the details of a specific solver is not a perfect solution (for example, I can think of code I wrote that was using Gurobi-specific features like solution pools etc) but I think it's our best bet.
Or-Tools looks great, though I've never used it. There is something open source as well but I forgot the name-- I assume that's less appealing anyway given your corporate constraints.
Recent I have actually implemented that glue to abstract solvers myself for a very project where I needed a very unusual interface that would also expose LP bases in a particular way, and I have to say building one such interface from scratch is very easy, so we could also go there if we can't figure out how to integrate Or-Tools or don't want to pay its cost in terms of bloating. I'm happy to contribute what I implemented (it supports GPLK, Gurobi and CPLEX).
One advantage of the in-house solution is that the code bloat will be minimal. Another is that we can treat it as an unstable implementation detail, and change it according to our needs. For example, if I were to port the code that uses solution pools, I would just be able to propose adding new methods to the interface.
About ABI incompatibility: I don't remember about CPLEX but I'm 99% sure we'll need to inject flags that disable glibcxx's new dual abi for strings as Gurobi's library files are not compiled with support for that.

Hi Gabriele, thanks this is great and good to know.
OR-Tools is the open-source one: (see https://github.com/google/or-tools). Internally, it's called something else, but it's so far API-compatible with OR-Tools. I think it's worth trying to push a bit more to get OR-Tools working with CMake publicly.  I'd be curious if you have any ideas on how to get it working.. it's quite similar to the libtorch setup you did (it has a CMake target so you can just plug it in, but seems to have a problem with dependencies).
@gabrfarina I always like reducing code bloat and dependencies, so I tend to like simpler in-house solutions (that are often easier to modify toward one's needs, and have complete control over) for that reason, so you definitely have my attention. That sounds like a lot of work, though!

About the open source project: I meant to say that there are other open source projects (besides Or-Tools) but I could not remember the name. Quickly looking into that now, I think the one I had in mind is part of the COIN-OR suite. But I think you would be reluctant to use something not vetted by Google?
I agree that pushing on the Or-Tools direction seems worth it for now. Building an abstraction over GLPK / Gurobi / CPLEX is not a lot of work though, at least in my experience. To retain the flexibility of using backend-specific features, I ended up settling for this type of architecture:

A LinearProgram interface that includes functions like SetObjective, SetLhsMatrix, SetTimeout, Solve, etc.
Three classes (GurobiLp, CplexLp, GlpkLp) that implement the LinearProgram interface and use the backend API. Often the implementation is just a forward of arguments.
Most algorithms expect a pointer to a LinearProgram and will fill it using the SetObjective/SetLhsMatrix etc, but calling Solve is the responsibility of the caller. This means that the caller can still call subclass-specific methods (for example, Gurobi-only methods) after the LP is filled, if necessary.

I have something similar to what @gabrfarina wrote in GTLib: https://github.com/aicenter/gtlib2/tree/master/LPsolvers

Alright, as promised. Code that is working internally is here: https://github.com/deepmind/open_spiel/tree/master/open_spiel/algorithms/ortools
Relevant commits:

Add OR-Tools: https://github.com/deepmind/open_spiel/commit/97a50f7b158a0d60d53fc04edaad55473504143c
Add CE solver: https://github.com/deepmind/open_spiel/commit/1a07c94a8d351693ff44025373a05479bddef26f

I managed to identify the source of the problem, and it is "the good old" absl. When I commented out linking the absl libraries with open_spiel_core and spiel.h, and changed the CMakeLists to only contain this:
set(BUILD_DEPS ON)
add_subdirectory(or-tools)
add_executable(simple_lp_solver_example simple_lp_solver_example.cc)
target_link_libraries(simple_lp_solver_example ortools::ortools)

it built and ran the example.

Awesome, thanks @michalsustr, that's progress! I will contact the OR-Tools (and maybe also absl teams) to see what they recommend as the right way to integrate OR-Tools into OpenSpiel.

I think it would be a good idea to make a version where we just link with a shared library of a specific version (v8?) and use provided headers, similarly to how it is done with libtorch. The OrTools repo has 700MB. It significantly slows down my IDE for many actions - cmake updates, code search, refactoring etc...

I agree...  I'd far prefer to just compile and link against a binary release of OR-Tools, and they do release binary distributions. By just grabbing compile and link flags from the example makefile, I managed to get it compiled and linked by just adding a few CMake options. But then it actually crashes when trying to run the matrix game example... and it happens deep within the internals of glop calling into an absl string function. When I contacted the OR-Tools devs, they told me that mixing C++17 and C++11 (which the OR-Tools binary distribution was compiled using) would not work.
I have since lost the flags but could easily get them again if anyone wants to try going down this route.

mixing C++17 and C++11 (which the OR-Tools binary distribution was compiled using) would not work.

This is the part that I find confusing, since the OrTools CMakeLists.txt specifies that it should be C++17.

Edit: v8 is C++17, and the previous v7.8 (which you used I guess) does not specify this. V8 was literally released 4 days ago, maybe it's worth trying again? :)

Nice! Ok, I will try it. In my reading of ABI incompatibilities, though, I think I did read that even within the same standard they different compilers (or version of compilers) could be ABI-incompable even for the same standard.
If it fails, I will add commented-out lines needed to run it externally.

Nice! Ok, I will try it. In my reading of ABI incompatibilities, though, I think I did read that even within the same standard that different compilers (or version of compilers) could still be ABI-incompable.
If it fails, I will add commented-out lines needed to run it externally. Thanks for letting us know.

Unfortunately, no luck. This time I can't even get it to compile. There are compilation errors with absl::StrFormat. My guess is that our Abseil versions are clashing (possibly ours is behind, and their code uses newer functions, but it's picking up ours). We can probably just upgrade to the same version, but unfortunately I don't have more time to look into it.
I will include a modified CMakeLists to show both options.

That sucks, but thanks for the update. At some point I will need to implement a value oracle, if we could get ortools working I will likely use it to write sqflp in c++.

Yes I really want SFLP on the C++ side. Can you say more about what you mean by value oracle? If it's to check the impl of games you could use the python SFLP implementation for now.
I suspect this absl fix will amount to being as easy as upgrading our version, which we should periodically do anyway. I will try it as soon as I hearfrom the OR-Tools team.

Ok.. good news! With an upgraded abseil, Option 2 (in the current CMakeLists-- updated just an hour ago) works! I will submit the updates and sync github later today or tomorrow.

Wow, that's awesome!! :)

Ok, here is the new CMakeLists.txt that runs with the OR-Tools v8: https://github.com/deepmind/open_spiel/commit/a6ad6ac5877686a1702a2d756edaed2b0aa8d870
(You will need to pull new changes from master, as we needed to upgrade our abseil version to get it working, which required some minor small things)

Yes, I managed to get it working!
At least the pre-compiled binary file. Somehow I have trouble compiling v8  from source now -- v7.8 was fine :) Anyway, I don't have time to investigate this now. I'd rather make sequence-form LP.

Great! I am happy to not support building OR-Tools from source for now, and come back to it when someone needs it. It could be useful to post the error or briefly describe it while it's fresh, for anybody who wants to look into it.
Looking forward to sequence-form LPs. @gabrfarina if you have specific design advice (keeping in mind that ideally we would like to re-use some of it for the non-zero sum / correlated equilibrium cases too) then please let us know.

It could be useful to post the error or briefly describe it while it's fresh, for anybody who wants to look into it.

It can't link with libpthread (among other things), but I do have it, also with the required functions (I checked nm /lib/x86_64-linux-gnu/libpthread.so.0 | grep "pthread_create" )
cat /home/michal/Code/open_or/build/CMakeFiles/CMakeError.log
Determining if the pthread_create exist failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_1670c/fast
/usr/bin/make -f CMakeFiles/cmTC_1670c.dir/build.make CMakeFiles/cmTC_1670c.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_1670c.dir/CheckSymbolExists.c.o
/usr/bin/cc   -fPIE   -o CMakeFiles/cmTC_1670c.dir/CheckSymbolExists.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
Linking C executable cmTC_1670c
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_1670c.dir/link.txt --verbose=1
/usr/bin/cc      CMakeFiles/cmTC_1670c.dir/CheckSymbolExists.c.o  -o cmTC_1670c
CMakeFiles/cmTC_1670c.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_1670c.dir/build.make:86: recipe for target 'cmTC_1670c' failed
make[1]: *** [cmTC_1670c] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_1670c/fast' failed
make: *** [cmTC_1670c/fast] Error 2

File /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_ba199/fast
/usr/bin/make -f CMakeFiles/cmTC_ba199.dir/build.make CMakeFiles/cmTC_ba199.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ba199.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=pthread_create -fPIE   -o CMakeFiles/cmTC_ba199.dir/CheckFunctionExists.c.o   -c /home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/share/cmake-3.14/Modules/CheckFunctionExists.c
Linking C executable cmTC_ba199
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_ba199.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=pthread_create    CMakeFiles/cmTC_ba199.dir/CheckFunctionExists.c.o  -o cmTC_ba199 -lpthreads
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_ba199.dir/build.make:86: recipe for target 'cmTC_ba199' failed
make[1]: *** [cmTC_ba199] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_ba199/fast' failed
make: *** [cmTC_ba199/fast] Error 2

Determining if the include file libunwind.h exists failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_aa698/fast
/usr/bin/make -f CMakeFiles/cmTC_aa698.dir/build.make CMakeFiles/cmTC_aa698.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_aa698.dir/CheckIncludeFile.c.o
/usr/bin/cc   -fPIE   -o CMakeFiles/cmTC_aa698.dir/CheckIncludeFile.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.c
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.c:1:10: fatal error: libunwind.h: No such file or directory
 #include <libunwind.h>
          ^~~~~~~~~~~~~
compilation terminated.
CMakeFiles/cmTC_aa698.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_aa698.dir/CheckIncludeFile.c.o' failed
make[1]: *** [CMakeFiles/cmTC_aa698.dir/CheckIncludeFile.c.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_aa698/fast' failed
make: *** [cmTC_aa698/fast] Error 2

Determining size of unsigned __int16 failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_b0d9e/fast
/usr/bin/make -f CMakeFiles/cmTC_b0d9e.dir/build.make CMakeFiles/cmTC_b0d9e.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_b0d9e.dir/HAVE___UINT16.c.o
/usr/bin/cc   -fPIE   -o CMakeFiles/cmTC_b0d9e.dir/HAVE___UINT16.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:23:31: error: expected ‘)’ before ‘__int16’
 #define SIZE (sizeof(unsigned __int16))
                               ^
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:25:12: note: in expansion of macro ‘SIZE’
   ('0' + ((SIZE / 10000)%10)),
            ^~~~
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:23:31: error: expected ‘)’ before ‘__int16’
 #define SIZE (sizeof(unsigned __int16))
                               ^
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:26:12: note: in expansion of macro ‘SIZE’
   ('0' + ((SIZE / 1000)%10)),
            ^~~~
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:23:31: error: expected ‘)’ before ‘__int16’
 #define SIZE (sizeof(unsigned __int16))
                               ^
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:27:12: note: in expansion of macro ‘SIZE’
   ('0' + ((SIZE / 100)%10)),
            ^~~~
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:23:31: error: expected ‘)’ before ‘__int16’
 #define SIZE (sizeof(unsigned __int16))
                               ^
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:28:12: note: in expansion of macro ‘SIZE’
   ('0' + ((SIZE / 10)%10)),
            ^~~~
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:23:31: error: expected ‘)’ before ‘__int16’
 #define SIZE (sizeof(unsigned __int16))
                               ^
/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:29:12: note: in expansion of macro ‘SIZE’
   ('0' +  (SIZE    % 10)),
            ^~~~
CMakeFiles/cmTC_b0d9e.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_b0d9e.dir/HAVE___UINT16.c.o' failed
make[1]: *** [CMakeFiles/cmTC_b0d9e.dir/HAVE___UINT16.c.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_b0d9e/fast' failed
make: *** [cmTC_b0d9e/fast] Error 2

/home/michal/Code/open_or/build/CMakeFiles/CheckTypeSize/HAVE___UINT16.c:
#include <sys/types.h>
#include <stdint.h>
#include <stddef.h>

#undef KEY
#if defined(__i386)
# define KEY '_','_','i','3','8','6'
#elif defined(__x86_64)
# define KEY '_','_','x','8','6','_','6','4'
#elif defined(__ppc__)
# define KEY '_','_','p','p','c','_','_'
#elif defined(__ppc64__)
# define KEY '_','_','p','p','c','6','4','_','_'
#elif defined(__aarch64__)
# define KEY '_','_','a','a','r','c','h','6','4','_','_'
#elif defined(__ARM_ARCH_7A__)
# define KEY '_','_','A','R','M','_','A','R','C','H','_','7','A','_','_'
#elif defined(__ARM_ARCH_7S__)
# define KEY '_','_','A','R','M','_','A','R','C','H','_','7','S','_','_'
#endif

#define SIZE (sizeof(unsigned __int16))
char info_size[] =  {'I', 'N', 'F', 'O', ':', 's','i','z','e','[',
  ('0' + ((SIZE / 10000)%10)),
  ('0' + ((SIZE / 1000)%10)),
  ('0' + ((SIZE / 100)%10)),
  ('0' + ((SIZE / 10)%10)),
  ('0' +  (SIZE    % 10)),
  ']',
#ifdef KEY
  ' ','k','e','y','[', KEY, ']',
#endif
  '\0'};

#ifdef __CLASSIC_C__
int main(argc, argv) int argc; char *argv[];
#else
int main(int argc, char *argv[])
#endif
{
  int require = 0;
  require += info_size[argc];
  (void)argv;
  return require;
}

Determining if the function dladdr exists failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_6364b/fast
/usr/bin/make -f CMakeFiles/cmTC_6364b.dir/build.make CMakeFiles/cmTC_6364b.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_6364b.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=dladdr -fPIE   -o CMakeFiles/cmTC_6364b.dir/CheckFunctionExists.c.o   -c /home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/share/cmake-3.14/Modules/CheckFunctionExists.c
Linking C executable cmTC_6364b
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_6364b.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=dladdr    -rdynamic CMakeFiles/cmTC_6364b.dir/CheckFunctionExists.c.o  -o cmTC_6364b
CMakeFiles/cmTC_6364b.dir/CheckFunctionExists.c.o: In function `main':
CheckFunctionExists.c:(.text+0x10): undefined reference to `dladdr'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_6364b.dir/build.make:86: recipe for target 'cmTC_6364b' failed
make[1]: *** [cmTC_6364b] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_6364b/fast' failed
make: *** [cmTC_6364b/fast] Error 2

Determining if the function get_static_proc_name exists in the unwind failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_d0b74/fast
/usr/bin/make -f CMakeFiles/cmTC_d0b74.dir/build.make CMakeFiles/cmTC_d0b74.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_d0b74.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=get_static_proc_name -fPIE   -o CMakeFiles/cmTC_d0b74.dir/CheckFunctionExists.c.o   -c /home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/share/cmake-3.14/Modules/CheckFunctionExists.c
Linking C executable cmTC_d0b74
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_d0b74.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=get_static_proc_name    CMakeFiles/cmTC_d0b74.dir/CheckFunctionExists.c.o  -o cmTC_d0b74 -lunwind
/usr/bin/ld: cannot find -lunwind
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_d0b74.dir/build.make:86: recipe for target 'cmTC_d0b74' failed
make[1]: *** [cmTC_d0b74] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_d0b74/fast' failed
make: *** [cmTC_d0b74/fast] Error 2

Determining if the function UnDecorateSymbolName exists in the dbghelp failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_8eb9e/fast
/usr/bin/make -f CMakeFiles/cmTC_8eb9e.dir/build.make CMakeFiles/cmTC_8eb9e.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_8eb9e.dir/CheckFunctionExists.c.o
/usr/bin/cc   -DCHECK_FUNCTION_EXISTS=UnDecorateSymbolName -fPIE   -o CMakeFiles/cmTC_8eb9e.dir/CheckFunctionExists.c.o   -c /home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/share/cmake-3.14/Modules/CheckFunctionExists.c
Linking C executable cmTC_8eb9e
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8eb9e.dir/link.txt --verbose=1
/usr/bin/cc  -DCHECK_FUNCTION_EXISTS=UnDecorateSymbolName    CMakeFiles/cmTC_8eb9e.dir/CheckFunctionExists.c.o  -o cmTC_8eb9e -ldbghelp
/usr/bin/ld: cannot find -ldbghelp
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_8eb9e.dir/build.make:86: recipe for target 'cmTC_8eb9e' failed
make[1]: *** [cmTC_8eb9e] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_8eb9e/fast' failed
make: *** [cmTC_8eb9e/fast] Error 2

Performing C SOURCE FILE Test HAVE_RWLOCK failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_6208f/fast
/usr/bin/make -f CMakeFiles/cmTC_6208f.dir/build.make CMakeFiles/cmTC_6208f.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_6208f.dir/src.c.o
/usr/bin/cc   -DHAVE_RWLOCK -fPIE   -o CMakeFiles/cmTC_6208f.dir/src.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c
Linking C executable cmTC_6208f
/home/michal/.python_envs/os/lib/python3.6/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_6208f.dir/link.txt --verbose=1
/usr/bin/cc  -DHAVE_RWLOCK    -rdynamic CMakeFiles/cmTC_6208f.dir/src.c.o  -o cmTC_6208f
CMakeFiles/cmTC_6208f.dir/src.c.o: In function `main':
src.c:(.text+0x24): undefined reference to `pthread_rwlock_init'
src.c:(.text+0x30): undefined reference to `pthread_rwlock_rdlock'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_6208f.dir/build.make:86: recipe for target 'cmTC_6208f' failed
make[1]: *** [cmTC_6208f] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_6208f/fast' failed
make: *** [cmTC_6208f/fast] Error 2

Source file was:

#define _XOPEN_SOURCE 500
#include <pthread.h>
int main(void)
{
  pthread_rwlock_t l;
  pthread_rwlock_init(&l, NULL);
  pthread_rwlock_rdlock(&l);
  return 0;
}

Performing C SOURCE FILE Test HAVE___DECLSPEC failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_ee4d6/fast
/usr/bin/make -f CMakeFiles/cmTC_ee4d6.dir/build.make CMakeFiles/cmTC_ee4d6.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ee4d6.dir/src.c.o
/usr/bin/cc   -DHAVE___DECLSPEC -fPIE   -o CMakeFiles/cmTC_ee4d6.dir/src.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c:2:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
 __declspec(selectany) int a;
 ^~~~~~~~~~
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c: In function ‘__declspec’:
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c:3:16: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
 int main(void) { return 0; }
                ^
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c:2:1: warning: type of ‘selectany’ defaults to ‘int’ [-Wimplicit-int]
 __declspec(selectany) int a;
 ^~~~~~~~~~
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c:2:27: error: declaration for parameter ‘a’ but no such parameter
 __declspec(selectany) int a;
                           ^
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.c:3:16: error: expected ‘{’ at end of input
 int main(void) { return 0; }
                ^
CMakeFiles/cmTC_ee4d6.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_ee4d6.dir/src.c.o' failed
make[1]: *** [CMakeFiles/cmTC_ee4d6.dir/src.c.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_ee4d6/fast' failed
make: *** [cmTC_ee4d6/fast] Error 2

Source file was:

__declspec(selectany) int a;
int main(void) { return 0; }

Performing C++ SOURCE FILE Test STL_NO_NAMESPACE failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_ee950/fast
/usr/bin/make -f CMakeFiles/cmTC_ee950.dir/build.make CMakeFiles/cmTC_ee950.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_ee950.dir/src.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS -DSTL_NO_NAMESPACE -fPIE   -o CMakeFiles/cmTC_ee950.dir/src.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.cxx:3:1: error: no template named 'vector'; did you mean 'std::vector'?
vector<int> t; int main() { }
^~~~~~
std::vector
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_vector.h:216:11: note: 'std::vector' declared here
    class vector : protected _Vector_base<_Tp, _Alloc>
          ^
1 error generated.
CMakeFiles/cmTC_ee950.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_ee950.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_ee950.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_ee950/fast' failed
make: *** [cmTC_ee950/fast] Error 2

Source file was:

#include <vector>
vector<int> t; int main() { }

Performing C++ SOURCE FILE Test HAVE_MSVC_TLS failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_82e75/fast
/usr/bin/make -f CMakeFiles/cmTC_82e75.dir/build.make CMakeFiles/cmTC_82e75.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_82e75.dir/src.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS -DHAVE_MSVC_TLS -fPIE   -o CMakeFiles/cmTC_82e75.dir/src.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/src.cxx:2:1: error: '__declspec' attributes are not enabled; use '-fdeclspec' or '-fms-extensions' to enable support for __declspec attributes
__declspec(thread) int tls;
^
1 error generated.
CMakeFiles/cmTC_82e75.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_82e75.dir/src.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_82e75.dir/src.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_82e75/fast' failed
make: *** [cmTC_82e75/fast] Error 2

Source file was:

__declspec(thread) int tls;
int main() { }

Determining if the include file ieeefp.h exists failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_ab33c/fast
/usr/bin/make -f CMakeFiles/cmTC_ab33c.dir/build.make CMakeFiles/cmTC_ab33c.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ab33c.dir/CheckIncludeFile.c.o
/usr/bin/cc   -fPIE   -o CMakeFiles/cmTC_ab33c.dir/CheckIncludeFile.c.o   -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.c
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.c:1:10: fatal error: ieeefp.h: No such file or directory
 #include <ieeefp.h>
          ^~~~~~~~~~
compilation terminated.
CMakeFiles/cmTC_ab33c.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_ab33c.dir/CheckIncludeFile.c.o' failed
make[1]: *** [CMakeFiles/cmTC_ab33c.dir/CheckIncludeFile.c.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_ab33c/fast' failed
make: *** [cmTC_ab33c/fast] Error 2

Determining if the include file cieeefp exists failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_aa46f/fast
/usr/bin/make -f CMakeFiles/cmTC_aa46f.dir/build.make CMakeFiles/cmTC_aa46f.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_aa46f.dir/CheckIncludeFile.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS  -fPIE   -o CMakeFiles/cmTC_aa46f.dir/CheckIncludeFile.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckIncludeFile.cxx:1:10: fatal error: 'cieeefp' file not found
#include <cieeefp>
         ^~~~~~~~~
1 error generated.
CMakeFiles/cmTC_aa46f.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_aa46f.dir/CheckIncludeFile.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_aa46f.dir/CheckIncludeFile.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_aa46f/fast' failed
make: *** [cmTC_aa46f/fast] Error 2

Determining if the isfinite exist failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_b3cd5/fast
/usr/bin/make -f CMakeFiles/cmTC_b3cd5.dir/build.make CMakeFiles/cmTC_b3cd5.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_b3cd5.dir/CheckSymbolExists.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS  -fPIE   -o CMakeFiles/cmTC_b3cd5.dir/CheckSymbolExists.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:11:19: error: address of overloaded function 'isfinite' does not match required type 'int'
  return ((int*)(&isfinite))[argc];
                  ^~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:577:3: note: candidate function
  isfinite(long double __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:573:3: note: candidate function
  isfinite(double __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:569:3: note: candidate function
  isfinite(float __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:585:5: note: candidate template ignored: could not match 'typename __gnu_cxx::__enable_if<__is_integer<_Tp>::__value, bool>::__type (_Tp)' against 'int'
    isfinite(_Tp __x)
    ^
1 error generated.
CMakeFiles/cmTC_b3cd5.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_b3cd5.dir/CheckSymbolExists.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_b3cd5.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_b3cd5/fast' failed
make: *** [cmTC_b3cd5/fast] Error 2

File /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
#include <cmath>
#include <cfloat>
#include <math.h>
#include <float.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef isfinite
  return ((int*)(&isfinite))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the _finite exist failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_ce16b/fast
/usr/bin/make -f CMakeFiles/cmTC_ce16b.dir/build.make CMakeFiles/cmTC_ce16b.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_ce16b.dir/CheckSymbolExists.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS  -fPIE   -o CMakeFiles/cmTC_ce16b.dir/CheckSymbolExists.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:11:19: error: use of undeclared identifier '_finite'
  return ((int*)(&_finite))[argc];
                  ^
1 error generated.
CMakeFiles/cmTC_ce16b.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_ce16b.dir/CheckSymbolExists.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_ce16b.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_ce16b/fast' failed
make: *** [cmTC_ce16b/fast] Error 2

File /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
#include <cmath>
#include <cfloat>
#include <math.h>
#include <float.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef _finite
  return ((int*)(&_finite))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the isnan exist failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_aa125/fast
/usr/bin/make -f CMakeFiles/cmTC_aa125.dir/build.make CMakeFiles/cmTC_aa125.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_aa125.dir/CheckSymbolExists.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS  -fPIE   -o CMakeFiles/cmTC_aa125.dir/CheckSymbolExists.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:11:19: error: address of overloaded function 'isnan' does not match required type 'int'
  return ((int*)(&isnan))[argc];
                  ^~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:631:3: note: candidate function
  isnan(long double __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:626:3: note: candidate function
  isnan(double __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:618:3: note: candidate function
  isnan(float __x)
  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/cmath:639:5: note: candidate template ignored: could not match 'typename __gnu_cxx::__enable_if<__is_integer<_Tp>::__value, bool>::__type (_Tp)' against 'int'
    isnan(_Tp __x)
    ^
1 error generated.
CMakeFiles/cmTC_aa125.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_aa125.dir/CheckSymbolExists.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_aa125.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_aa125/fast' failed
make: *** [cmTC_aa125/fast] Error 2

File /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
#include <cmath>
#include <cfloat>
#include <math.h>
#include <float.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef isnan
  return ((int*)(&isnan))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the _isnan exist failed with the following output:
Change Dir: /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_d065b/fast
/usr/bin/make -f CMakeFiles/cmTC_d065b.dir/build.make CMakeFiles/cmTC_d065b.dir/build
make[1]: Entering directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_d065b.dir/CheckSymbolExists.cxx.o
/usr/bin/clang++    -Wno-everything -DBUILD_WITH_PYTHON -DBUILD_WITH_ORTOOLS  -fPIE   -o CMakeFiles/cmTC_d065b.dir/CheckSymbolExists.cxx.o -c /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:11:19: error: use of undeclared identifier '_isnan'; did you mean '__isnan'?
  return ((int*)(&_isnan))[argc];
                  ^~~~~~
                  __isnan
/usr/include/x86_64-linux-gnu/bits/mathcalls-helper-functions.h:36:20: note: '__isnan' declared here
__MATHDECL_1 (int, __isnan,, (_Mdouble_ __value)) __attribute__ ((__const__));
                   ^
1 error generated.
CMakeFiles/cmTC_d065b.dir/build.make:65: recipe for target 'CMakeFiles/cmTC_d065b.dir/CheckSymbolExists.cxx.o' failed
make[1]: *** [CMakeFiles/cmTC_d065b.dir/CheckSymbolExists.cxx.o] Error 1
make[1]: Leaving directory '/home/michal/Code/open_or/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_d065b/fast' failed
make: *** [cmTC_d065b/fast] Error 2

File /home/michal/Code/open_or/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:
/* */
#include <cmath>
#include <cfloat>
#include <math.h>
#include <float.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef _isnan
  return ((int*)(&_isnan))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Looking forward to sequence-form LPs. @gabrfarina if you have specific design advice (keeping in mind that ideally we would like to re-use some of it for the non-zero sum / correlated equilibrium cases too) then please let us know.

Sorry for the late reply @lanctot and for not getting around to giving feedback yet to your PRs as you were asking, @michalsustr. Great to hear there was positive movement around integrating OR-Tools as a dependency.
I've given some thought as to what we learnt over time building our own infrastructure (which mostly builds around the online learning = "each player has its own independent decision space" point of view instead of dealing with a game tree), and what choices would simplify porting some of the algorithms we've been working on as easily as possible:

One of the major design decisions that we had to revert at some point was that of always carrying around and looking up sequences in hash maps. Hash maps are way more expensive than we wanted, and while the interface was very user friendly, it was not very performance friendly. So, at some point we decided to make a change that ended up improving scalability a lot: sequences are mapped to contiguous integers. Information sets are assigned a contiguous block of sequence IDs, and the sequence numbering is done so that deeper sequences get assigned lower IDs. The empty sequence is the last sequence. This guarantees that a lot of optimization primitives boil down to a very cache friendly traversal of an array. This applies for example to proximal operators, projection algorithms, regret minimizers, etc.

The user-friendly conversion between (infoset, action) pairs and sequence IDs is handled outside from the optimization algorithms. In other words, all algorithms do not depend on the game tree and do not deal with pretty representations of sequences --- just integers.

In terms of interface, we have two main objects to represent the decision spaces of the agents:

Infoset represents an information set, and contains three integers: parent_seq_id, start_seq_id and end_seq_id. The first is the parent sequence of the information set, while the last two sequence IDs represent the sequence id of the first and last action at the information set. Because sequences assigned to an infoset are contiguous, we don't need to store all intermediate sequence IDs. This is really all there is to it (Range is just an iterable to make looping frictionless):
struct Infoset {
  SequenceId start_seq_id;
  SequenceId end_seq_id;
  SequenceId parent_seq_id;

  size_t NumSequences() const;
  Range<SequenceId> AllSequenceIds() const;
};

Treeplex represents the whole decision space of the agent, so effectively the collection of all of the player's infosets. It is a vector of Infoset objects, where the infosets are stored in a bottom-up order so that all traversals are in cache. Here is a subset of the methods that we currently expose for Treeplex (you can think of Slice as an array_view akin to string_view):
class Treeplex {
  PlayerId Player() const;
  size_t NumInfosets() const;
  size_t NumSequences() const;
  size_t MaxInfosetSize() const;
  SequenceId EmptySequenceId() const;
  Range<SequenceId> AllSequenceIds() const;

  const Infoset& GetInfoset(const InfosetId& infoset_id) const;
  Slice<Infoset> AllInfosets() const;
  Range<InfosetId> AllInfosetIds() const;
  Slice<InfosetId> InfosetsWithParentSeq(const SequenceId& sequence_id) const;

  // Returns `None` if the sequence is the empty sequence.
  std::optional<InfosetId> InfosetIdForSequence(
      const SequenceId& sequence_id) const;
  std::pair<FloatType, SfStrategy> BestResponse(
      TreeplexVector grad /* consumed */) const;
  bool IsSfStrategy(const TreeplexVector& vector) const;
};

TreeplexVectors are simply arrays that can be easily indexed by SequenceIds, plus a bunch of zero-cost sugar to make debugging easier. SfStrategy is a wrapper around TreeplexVector that also performs additional opt-in checks.
Treeplexes are usually passed around behind a shared pointer. The pretty conversion between sequence IDs and (infoset, action) pairs in the original game tree could be done by the Treeplex object (though like I was saying ideally algorithms would not rely on it).

To make debugging common mistakes easier, we have an opt-in, debug-time instrumentation of SequenceIds so that they carry around what Treeplex they came from to loudly fail if we accidentally use sequences for a player to index quantities of some other player.

On top of Treeplex there are more sophisticated objects to handle correlation / coordination or views over subtrees to compute the EFCE / EFCCE / NFCCE gap and write LPs for correlated solution concepts in a way that's maintainable and that won't make people cry (took a while to get there). But once the Treeplex interface is in place with fast support for the operations above, everything flows pretty nicely. We've been using this structure for a couple of years at this point and have been pretty happy with how future proof it managed to be.

Let me know if this is helpful --- I'm happy to get into more details and help out if you are interested in pursuing something similar for OpenSpiel.

Hi @lanctot ,
sorry for my late reply! (I am currently a bit busy with my Master's thesis and unfortunately also my laptop with some non-backed up parts of the minimax-Q-learning code was stolen^^)
In our code for Goofspiel we were only making use of the LP-solvers on the python side and I don't have experience with C++ LP-solvers. But it is great to hear that those are now available! I hope I'll find some time soon to finish the minimax-Q (in case this hasn't already been implemented :))

Thank you @gabrfarina for a very detailed comment! I think all you described can be built (integrated) easily with the code I've written, if you're interested in going this path. I will be posting chunks (and further extensions) of #393 in the coming weeks.

Information sets are assigned a contiguous block of sequence IDs, and the sequence numbering is done so that deeper sequences get assigned lower IDs.

Interesting optimization. You preconstruct the whole treeplex, is that correct? Because a dynamical construction would require shifting of sequence IDs to satisfy the "deeper sequence has lower ID" constraint, which will break internal references to those IDs. I am asking about dynamical construction because at some point I'd like to port double-oracle sequence form LP (I've made SF-LP already) and I'm not sure if that works with static construction (but it probably does now I think of it more).
Also, by "sequence" you mean the "standard" definition, i.e. a list of actions taken by the player so far, correct?

const Infoset& GetInfoset(const InfosetId& infoset_id) const;

infoset_id is just an index of the Infoset in the collection (vector) of all infosets?

To make debugging common mistakes easier, we have an opt-in, debug-time instrumentation of SequenceIds so that they carry around what Treeplex they came from to loudly fail if we accidentally use sequences for a player to index quantities of some other player.

In non-debug mode they are just int right?

I think all you described can be built (integrated) easily with the code I've written [..]

Nice!

Interesting optimization. You preconstruct the whole treeplex, is that correct? Because a dynamical construction would require shifting of sequence IDs to satisfy the "deeper sequence has lower ID" constraint [..]

We do! Note that you could also reorder the sequences so that lower IDs are given to the top, and then iterate from right to left in memory when you need to do a bottom-up pass. That is also cache friendly.

Also, by "sequence" you mean the "standard" definition, i.e. a list of actions taken by the player so far, correct?

Yes, though I find it more productive to think about it as an (infoset, action) pair. Since the underlying model is a tree, the two points of view are equivalent.

infoset_id is just an index of the Infoset in the collection (vector) of all infosets?

Yes, plus the opt-in instrumentation I was mentioning.

In non-debug mode they are just int right?

I believe they are size_ts.

TreeplexVectors are simply arrays that can be easily indexed by SequenceIds

But they are not contiguous, right? What I mean, you can't have a SequenceId to have values as 0 .. N-1 where N is the number of terminal nodes, because the treeplex may not be "balanced" (all the player's action sequences do not have the same length, for example folding early in Poker).

I think all you described can be built (integrated) easily with the code I've written [..]

Nice!

Now that I think of I estimate it ~200 lines of code, if we make a slight "relaxation" of what a SequenceId is :) Can it contain edges coming from observation nodes?

But they are not contiguous, right? What I mean, you can't have a SequenceId to have values as 0 .. N-1 where N is the number of terminal nodes, because the treeplex may not be "balanced" (all the player's action sequences do not have the same length, for example folding early in Poker).

Yes, the indices form a contiguous interval starting at 0. Also, the total number of sequences is not the number of terminal nodes: for each information set, you have as many sequences as actions at that information set. So, the total number of sequences is larger than just the terminal sequences. So, the total number of sequences is the sum over all information sets of the number of actions at the information set. I'm not sure I understood your comment about "balanced" treeplexes?

If we make a slight "relaxation" of what a SequenceId is :) Can it contain edges coming from observation nodes?

That would be very nonstandard given what sequence means in the literature (starting form von Stengel 1996). Things like the sequence-form polytope, the von Stengel-Forges polytope for correlation, the dilated entropy regularizer, etc as written in the literature all break if you redefine sequences to keep into account observations. Also, you would end up overcounting sequences that way, when in reality you can get away with less (which helps scalability). Of course, you could fix all that breaks, but my two cents here are that unless there is a compelling need, I would not create an unnecessary barrier by deviating from the literature.
In order to implement optimization algorithm, you only usually need to reason about your decision space, which corresponds to your actions. Observations are automatically kept into account by the fact that you have more information sets based on what you observed. I hope that makes sense 🙂

How do you "connect" terminal histories of the game, where the players receive the utilities? Does the TreeplexVector span all of the terminals of the game, so each player has the same size of the TreeplexVector? Then you could use something like std::vector<std::vector<SequenceId>> xs to specify that at history xs[i] the player p has a SequenceId of xs[i][p].

I'm not sure I understood your comment about "balanced" treeplexes?

Sorry, I shouldn't be writing comments past midnight :)) I was thinking about an effective (and simple) algorithm that would put these labels for the sequences. If all terminals had the same sequence length, you could just use std::sort on them.

That would be very nonstandard given what sequence means in the literature [..]

Yes. The motivation was just that you could use one indexing (a "coodinate system") to address both sequences and infosets, which would effectively correspond to numbering all the nodes within an infostate tree. I have a use-case exactly for this, and I thought it could be applied here as well. But I agree it is non-standard and it would be better to keep them separate.

I hope that makes sense :slightly_smiling_face:

Yes it does. I'm trying to think of an implementation that would ideally mimic the interface you mentioned before and also work for my use-case, that's why I ask so many questions :)

How do you "connect" terminal histories of the game, where the players receive the utilities?

Ah, I think I see your question. So, as a general comment, in online optimization you don't care as much as to "how losses/utilities are computed", usually: you just want to build an algorithm that can handle any sequence of such losses. This being said, to connect online optimization to game theory, of course you need to instantiate algorithm for particular sequences of losses depending on the specific application at hand. For example, in a bilinear saddle point like Nash equilibrium or TMECor for a team---a problem of the form min_x max_y x^T A y---you might want to use losses Ay^t and A^T x^t at all times t, where x^t and y^t are the strategies output by the online optimization algorithms set up for the two players and A is the payoff matrix of the game.
In order to compute a loss of the form A y^t, you can either do a full pass on the game tree, or you might want to precompute the sequence-form payoff matrix A. We usually do the latter.

Does the TreeplexVector span all of the terminals of the game, so each player has the same size of the TreeplexVector?

A TreeplexVector is just a real vector with as many coordinates as sequences for the player. Not just terminal sequences, all sequences. A sequence form strategy is represented as a TreeplexVector, a loss vector of the form A y^t is represented as a TreeplexVector, etc. We have a wrapper so that we can index using a SequenceId, as well as do other things that come up often in practice, like extracting a subvector of entries relative to one information set only.

[..] you could use something like std::vector<std::vector<SequenceId>> xs to specify that at history xs[i] the player p has a SequenceId of xs[i][p].

That would work. How were you planning to keep track of the association between InfosetIds and InfostateString (or InfostateTensor)? If you knew that LegalAction returns actions always in the same order at all nodes that belong to the same infoset, perhaps it would be more compact to only keep track of the InfostateString <-> InfosetId bijection and convert between sequences and (InfostateString, ActionString) pairs implicitly/on the fly?

[..] full pass on the game tree or [..] precompute the sequence-form payoff matrix A [..]

Aren't both quite inefficient? Isn't A a huge sparse matrix of a dimension |S_x| x |S_y| (S being the set of sequences for given player)?
I made a representation that needs to use two vectors of a size upper bounded by max(|S_x|, |S_y|). One vector are the utilities, the other is a permutation indexing which changes the "coordinate system" of the computed utilities to the other player. An example.
This is why I was talking about the sequences only restricted to terminals, because values for any other prefixes can be computed by the BottomUp traversal.

How were you planning to keep track of the association between InfosetIds and InfostateString?

Currently the InfostateNode represents an infostate and contains the infostate string. All that would be needed is to have a std::vector<InfostateNode*> to store appropriate pointers for the InfosetId -> Infoset.

If you knew that LegalAction returns actions always in the same order at all nodes that belong to the same infoset

Yes, they are sorted: #41

Aren't both quite inefficient?

That is relative. CFR, mirror descent, EGT, etc they all need to compute the full gradient vectors of the form Ay^t that I was mentioning.
Depending on the algorithm, there other other things you could do as well, for example constructing an unbiased estimator of Ay^t instead of the exact gradient. That leads to MCCFR and other stochastic regret minimization methods. Those stochastic methods still guarantee convergence to the saddle point with high probability---it's a pretty general and well known fact in online learning, though for game solving in particular it seems like prior analyses were more opaque than needed. Christian and I wrote about it a few months ago for ICML.

Isn't A a huge sparse matrix of a dimension |S_x| x |S_y| (S being the set of sequences for given player)?

It is! Like you wrote, it is sparse. The number of nonzeros is upper bounded by the number of terminal nodes (not sequences) of the original game tree.

One vector are the utilities, the other is a permutation indexing which changes the "coordinate system" of the computed utilities to the other player.

I'm not sure I follow. When you said "I made a representation" did you mean a representation of the payoff matrix? Because in the worst case the payoff matrix has a number of nonzeros that is big Omega(|S_x| * |S_y|).

[..] sequences only restricted to terminals, because values for any other prefixes can be computed by the BottomUp traversal.

What you write is technically correct, but the extra sequences are important and you should not remove them just because they are linearly dependent.  The first example that comes to mind is the sequence-form constraints: if you don't have the extra sequences, the constraints become pretty painful. Another example is the dilated entropy regularizers of Hoda et al., where the proof of strong convexity and the algorithm to compute the Fenchel congujate all hinge on the presence of the extra sequences.
When moving to the "correlated" sequence form, those extra sequences become even more important.
By the way, the payoff matrix might have nonzero entries that correspond to non-terminal sequences. Not sure if it's relevant right now, but just a thing to keep an eye out for.

All that would be needed is to have a std::vector<InfostateNode*> to store appropriate pointers for the InfosetId -> Infoset

Yeah that sounds reasonable. How about the mapping from Infoset(String) to InfosetId?

Yes, they are sorted [..]

Ok then I think it makes more sense to only store the Infoset <-> InfosetId bidirectional mapping somehow, and convert SequenceIds to and from (Infoset, Action) pairs on the fly.

Thanks for the comments! I think I now have a pretty clear picture of what needs to be done. I will try to make the time for a PR in the coming few days. Btw I've sent you also an e-mail to @cs.cmu.edu

Like you wrote, it is sparse.

Do you use Eigen?

Thanks for the comments! I think I now have a pretty clear picture of what needs to be done. I will try to make the time for a PR in the coming few days. Btw I've sent you also an e-mail to @cs.cmu.edu

Awesome! Let me know if you need a hand or anything. I'm also happy to jump in a zoom meeting or something like that if anything comes up.

Do you use Eigen?

Not by default. The default representation of the payoff matrix is just as a vector of PayoffMatrixEntry, defined as
struct PayoffMatrixEntry {
  PlayerMap<SequenceId> sequence_ids;
  PlayerMap<FloatType> payoffs;
  FloatType chance_factor;
};

PlayerMap<T> is just a map from PlayerId to T, with the added sugar that you can index by PlayerId, and iterate over all the (PlayerId, T) pairs it contains. Internally it is just a (stack-allocated) array.
In my experience, it is important to distinguish between the payoffs at a particular leaf, and the chance_factor at that leaf, defined as the product of the chance actions on the path from the root to the leaf. That is because some algorithms require to shift all payoffs (for example, the Lemke-Howson algorithm for general-sum Nash equilibrium in 2-player games). It would be useful if you could do the same I think.
In order to get the payoff matrix of a player, you would multiply the payoffs for that player times the chance factors.

I think it will be better seen through the code, which I wrote specifically with depth-limited CFR in mind.

Sounds good. I'm still a bit confused as to how you can go from something that is quadratic in the worst case to something that is linear in the worst case, but maybe there was a misunderstanding.

I feel like the conversation has mostly been resolved, closing for now. Please re-open if you want to continue it!


Formulas no longer working when querying across ManyToMany relationship

Describe the bug
It seems that MikroORM breaks formulas which are no longer pulled across a ManyToMany relationship. See the example below.
Reproduction
Consider the following 3 entities:
@Entity()
export class User {
  @Property({ type: t.text, nullable: true })
  firstName?: string;

  @Property({ type: t.text, nullable: true })
  lastName?: string;

  @Formula((alias) => `CONCAT(${alias}.first_name, ' ', ${alias}.last_name)`)
  fullName?: string;

  @Formula((alias) => `LOWER(${alias}.email)`)
  emailLower?: string;
}

@Entity()
export class PageRevision {
  @ManyToMany({
    entity: () => User,
    inversedBy: (user) => user.pageRevisions,
    owner: true,
    pivotEntity: () => PageRevisionUser,
  })
  users?: Collection<User> = new Collection<User>(this);
}

@Entity()
export class PageRevisionUser {
  @ManyToOne(() => PageRevision, {
    primary: true,
    fieldName: 'page_revision_id',
    onUpdate: () => 'cascade',
    onDelete: 'cascade',
  })
  pageRevision: PageRevision;

  @ManyToOne(() => User, {
    primary: true,
    fieldName: 'user_id',
    onUpdate: () => 'cascade',
    onDelete: 'cascade',
  })
  user: User;
}

Now consider the following query:
const revision = await this.pageRevisionRepository.findOne(where, {
  populate: ['users'],
});

Below is the generated SQL query:
-- Mikro ORM 5.9.x
-- Notice the two formulas
SELECT
	"u0".*,
	CONCAT("u0".first_name, ' ', "u0".last_name) AS "full_name",
	LOWER("u0".email) AS "email_lower",
	"p1"."user_id" AS "fk__user_id",
	"p1"."page_revision_id" AS "fk__page_revision_id"
FROM
	"user" AS "u0"
	LEFT JOIN "page_revision_user" AS "p1" ON "u0"."id" = "p1"."user_id"
WHERE
        "p1"."page_revision_id" in('018e1bb2-a8e0-81f6-c008-6811fe9d6c63')

-- Mikro ORM 6.x.x
-- Note that formulas are no longer included
SELECT
	"u1".*,
	"p0"."user_id" AS "fk__user_id",
	"p0"."page_revision_id" AS "fk__page_revision_id"
FROM
	"page_revision_user" AS "p0"
	INNER JOIN "public"."user" AS "u1" ON "p0"."user_id" = "u1"."id"
WHERE
	"p0"."page_revision_id" in('018e1bb2-a8e0-81f6-c008-6811fe9d6c63')

What driver are you using?
@mikro-orm/postgresql
MikroORM version
6.1.8
Node.js version
20.x
Operating system
MacOS
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Slack.
[X] The provided reproduction is a minimal reproducible example of the bug.

Let me know if you need a minimum reproducible example. I can put something together but had to significantly simplify the above classes because it comes from a fairly complex app with many more properties. I was hoping that it might be an obvious use case to replicate and fix.

Let me know if you need a minimum reproducible example

I always need one :)

Yes, we are on v6 and yes it does compile. I'll put together a repro.

Yes, we are on v6 and yes it does compile.

Your screenshot is not the same as what you have put into the issue description. Note the deleteRule (on screenshot) vs onDelete (in the issue).

Sorry, I simplified the entities and I must have based them on a different branch or something. What's in the screenshot is right.

Here is a passing repro, let's reopen with a failing one:
import { Entity, MikroORM, Property, ManyToOne, Collection, ManyToMany, Formula, PrimaryKey } from '@mikro-orm/sqlite';
import { mockLogger } from '../../helpers';

@Entity()
class User {

  @PrimaryKey()
  id!: number;

  @Property({ nullable: true })
  firstName?: string;

  @Property({ nullable: true })
  lastName?: string;

  @Property({ nullable: true })
  email?: string;

  @Formula(alias => `CONCAT(${alias}.first_name, ' ', ${alias}.last_name)`)
  fullName?: string;

  @Formula(alias => `LOWER(${alias}.email)`)
  emailLower?: string;

}

@Entity()
class PageRevision {

  @PrimaryKey()
  id!: number;

  @ManyToMany({
    entity: () => User,
    pivotEntity: () => PageRevisionUser,
  })
  users = new Collection<User>(this);

}

@Entity()
class PageRevisionUser {

  @ManyToOne(() => PageRevision, { primary: true })
  pageRevision!: PageRevision;

  @ManyToOne(() => User, { primary: true })
  user!: User;

}

let orm: MikroORM;

beforeAll(async () => {
  orm = await MikroORM.init({
    dbName: ':memory:',
    entities: [PageRevision, User, PageRevisionUser],
    debug: true,
  });
  await orm.schema.refreshDatabase();
});

afterAll(async () => {
  await orm.close(true);
});

test('GH #5319', async () => {
  const mock = mockLogger(orm);
  await orm.em.findAll(PageRevision, { populate: ['users'] });
  expect(mock.mock.calls[0][0]).toMatch('select `p0`.*, `u1`.`id` as `u1__id`, `u1`.`first_name` as `u1__first_name`, `u1`.`last_name` as `u1__last_name`, `u1`.`email` as `u1__email`, CONCAT(`u1`.first_name, \' \', `u1`.last_name) as `u1__full_name`, LOWER(`u1`.email) as `u1__email_lower` ' +
    'from `page_revision` as `p0` ' +
    'left join `page_revision_user` as `p2` on `p0`.`id` = `p2`.`page_revision_id` ' +
    'left join `user` as `u1` on `p2`.`user_id` = `u1`.`id`');
});

Just following up on this one. I don't know what happened but this issue seems resolved and formulas work fine again in the latest release 6.2.1. We skipped the last few minor releases due to various subtle issues introduced in v6 that we had to work around in various ways but this last release seems to have fixed at least some of them, thank you!


[CARBONDATA-2324] Support config ExecutorService in search mode

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?

[ ] Any backward compatibility impacted?

[ ] Document update required?

[ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4893/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4362/

retest this please

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4966/

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4973/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/4979/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/4425/

LGTM


Error: Cask 'gitter' definition is invalid: Bad header line: parse failed

trying to install gitter, getting this error:
Error: Cask 'gitter' definition is invalid: Bad header line: parse failed

This error is not a bug, it's caused by an update to how homebrew-cask works. Details can be found here.
To fix the issue, please run:
brew update; brew cleanup; brew cask cleanup
brew uninstall --force brew-cask; brew update

If you are still having problems, open an issue with the appropriate template

Got it thanks.


Sync with upstream v0.19.0

What does this change?

Syncs with the upstream repository apache/thrift 0.19.0 branch
Removes reserved keywords, similar to 3bae6d21037691e1bf4fec84f16af0bd6bfa925b. See https://github.com/guardian/french-thrift/pull/12#issuecomment-1804240046.

How to test

[x] Before merging, create a prerelease from this branch (here)
[ ] Create a branch in Bridget to check this prerelease works

Set the validate-thrift action's Dockerfile to use the prerelease: https://github.com/guardian/bridget/blob/main/.github/actions/validate-thrift/Dockerfile#L3
Disable the paths filter  for the validate-thrift workflow so it runs for the test Pull Request

Further details
I followed the directions outlined in the GitHub documentation for syncing a fork with the upstream repository from the command line.

It looks like recent releases of french-thrift in fact were just "thrift", in that they did not apply the commits that unreserved the reserved keywords. v0.9.3 is the most recent version of this library which includes the advertised features.
Subsequent releases have dropped the removal of reserved keywords, and introduced things like async Swift generators. It's possible that later versions of thrift include async Swift generators. If so, consumers of french-thrift > v0.9.3 could probably just use thrift directly.

Closing in favour of #13


Does spring-cloud-sleuth library support protobuf?

Question
As I can conclude from documentation your library can track HTTP/HTTPS requests. But does it support protobuf protocol?

which protobuf library are you using for rpc?
On Wed, 26 Sep 2018, 04:27 Oleksandr Sankin<EMAIL_ADDRESS>wrote:

Question
As I can conclude from documentation your library can track HTTP/HTTPS
requests. But does it support protobuf protocol?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/spring-cloud/spring-cloud-sleuth/issues/1099, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD6191CJnkMAYpcEcgfwVMtFAe4dlD2ks5uezppgaJpZM4W6JEf
.

This one https://github.com/protocolbuffers/protobuf

It doesn't currently support that, but you can raise an issue on Brave. If
a few others are interested, we can make something. Meanwhile it might be
similar in nature to other RPC frameworks. Let's carry the discussion here
https://github.com/openzipkin/brave/issues
On Wed, Sep 26, 2018 at 10:32 AM Oleksandr Sankin<EMAIL_ADDRESS>wrote:

This one https://github.com/protocolbuffers/protobuf
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/spring-cloud/spring-cloud-sleuth/issues/1099#issuecomment-424737452,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD610Y4FctGw_Ihx3aRHKxEg99tnB_rks5ue4_kgaJpZM4W6JEf
.

I'll close the issue then


Add functionality to push all switchbots with ESP32 BOOT button

I have to admit I'm not hugely familiar with cpp yet but here is a PR that's working for me to use the ESP32 BOOT button to toggle all switchbots :)
Useful for me so thought I'd share

lol. I barely know c++. It is not one of the languages I work with daily. But knew just enough to get this project going
that's a neat idea and I could probably add something like that, but I would probably make a few code changes. Good work though
things I would change:

create another list for bots to control with the esp32 button (instead of all bots)
add the command to control the bot to the queue of commands instead of sending the command directly. This will ensure the retry logic works etc and doesn't interfere with MQTT commands coming in. Like this...

struct QueueCommand queueCommand;
queueCommand.payload = "OFF";
queueCommand.topic = ESPMQTTTopic + "/control";
queueCommand.device = aDevice;
queueCommand.disconnectAfter = true;
queueCommand.priority = false;
queueCommand.currentTry = 1;
commandQueue.enqueue(queueCommand);

instead of ..
 controlMQTT(aDevice, "OFF", true);

This could also technically apply to curtains

I am working on getting the v7.0 release, which is here https://github.com/devWaves/SwitchBot-MQTT-BLE-ESP32/tree/v7alpha
The only remaining issue in v7.0 is related to mesh stuff I added, but it includes a bunch of other changes. So I will be releasing it soon either way
Once I get v7.0 out, I will try to take a look a this
thanks for sharing

@devWaves thanks man! I might come back to this when you release v7 if you haven't done your own implementation :D
Thanks to you I just successfully ripped this code apart and created a working ESP32 keypad to arm/disarm my HASS alarm :) thank you!!!

@devWaves updated it :)


Fix: native-image generation fails because of newer lucene version

In this PR I mainly focused on those points:

[x] Reenabled native tests in CI, so errors in native-image generation will be more visible in future:
[x] It seems, that the update to lucene 9.5.0 broke the "Unmap"-Workaround, that lead to build failures like #140
[x] The field type "InetAddressPoint" has to be initialized at runtime. I don't know exactly why this is an error NOW, but either the lucene update introduced it, or the GraalVM forces us to do so now (currently I am using GraalVM 22.3.0 with Java 17)

-> Maybe we should update the README.MD, so it gets clear, that the extension can be run with a newer Java und GraalVM version?

I'll have a look at it tomorrow, so I can verify the correct functionality in our production project.

@dainiusjocas - maybe you could have a look at this PR? :)


Add checkpoint_callback_best_model_monitor to ModelCheckpoint

Motivation
If I have a directory with many checkpoints, possibly belonging to runs with different monitors, I don't know which metric was tracked by each checkpoint.
Pitch
ckpt = torch.load("saved_model.ckpt")
assert hasattr(ckpt, "checkpoint_callback_best_model_monitor"

Alternatives
Guess If I included the monitor in the checkpoint name, I could parse it to know. But seems much more convenient to add the property.

another alternative: you can also add that to checkpoint itself in on_save_checkpoint.

Do you mean by overriding it in my LightningModule?
It is an ugly solution because it does not know about any ModelCheckpoints used. Only the trainer knows about them.
So I would need to instantiate my LightningModule with an added checkpoint_callback_monitor parameter in order to do:
def on_save_checkpoint(self, checkpoint):
    checkpoint["checkpoint_callback_best_model_monitor"] = self.checkpoint_callback_monitor

@carmocca here

If you guys agree with this issue, I can work on the PR

I think it makes sense to add monitor especially since best_model_score is already included. Lightning should track what that score actually represents 😃

Do you mean by overriding it in my LightningModule?

na.. not the LightningModule but ModelCheckpoint, I know it's ugly haha..
IMO I think it's good to have the monitor as well. But need to check whether it is required to load it back or not in on_load_checkpoint.


Use password input for sensitive information

This PR depends on https://github.com/apache/brooklyn-server/pull/185

LGTM; merging.


PDF not rending/showing with some mobile browsers (Xiaomi Mi Browser, Samsung Internet Browser, ...)

Attach (recommended) or Link to PDF file here:
https://canalcomercio.es/oferta/contigo/contigo-tenerife-comida-saludable#menu You can see the pdf here. Attached is the pdf.

Configuration:

Web browser and its version:
Happens with two different system that we tested:
-> Xiaomi mix s2 with MIUI 12.0.1 and MI Browser 12.X
-> Samsung S 10 with Samsung Internet Browser

Operating system and its version: Android 10

PDF.js version: 2.7.256 (loading from https://mozilla.github.io/pdf.js/build/pdf.js)

Is a browser extension: NO

Steps to reproduce the problem:

Visit the think below with any of the devices listed before (in our tests we use the browser embeded in the QR code reader from the phone, that opens the mobile's browser)

What is the expected behavior? (add screenshot)
The pdf is shown

What went wrong? (add screenshot)
The pdf is not shown

I don't have the browser logs for this or the exact version of the browser, but I'll try to collect the files and posted. This seems to happen to modern in browser mobile.

https://canalcomercio.es/oferta/contigo/contigo-tenerife-comida-saludable#menu You can see the pdf here. Attached is the pdf.

That seems to link to an HTML page, rather than directly to a PDF file (it's somewhat difficult to navigate that page if you don't speak, what looks like, Spanish). Please make sure that you always attach the actual PDF file here.
Also, please note https://github.com/mozilla/pdf.js/blob/master/.github/CONTRIBUTING.md (emphasis mine):

If you are developing a custom solution, first check the examples at https://github.com/mozilla/pdf.js#learning and search existing issues. If this does not help, please prepare a short well-documented example that demonstrates the problem and make it accessible online on your website, JS Bin, GitHub, etc. before opening a new issue or contacting us in the Matrix room -- keep in mind that just code snippets won't help us troubleshoot the problem.

Web browser and its version:
Happens with two different system that we tested:
-> Xiaomi mix s2 with MIUI 12.0.1 and MI Browser 12.X
-> Samsung S 10 with Samsung Internet Browser

Did you test with any other, more "standard", browsers such as e.g. Firefox and/or Chrome?
Please note that it's much more likely that this is an issue with the platform/browser itself, rather than the PDF.js library, and as such it's unfortunately difficult to see that there's much that can be done about this here.

Hi,
Here is the pdf version of the file: https://canalcomercio.es/ServidorDocumentos/0/OFERTAS/2493/20201029083333786CONTIGO2convertido.pdf
This doesn't seems to be related to the PDF itself, it's about the pdf.js and how it's shown and the browser Miui Browser and other mobile
I've tested viewing the code with this phone and used the chrome debugger:
=> https://www.whatsmybrowser.org/b/LLMHZ
=> Miui Browser 71 - V12.4.1-g
Code snippet:

<script type="text/javascript">

    // Javascript to enable link to tab
    var url = document.location.toString();
    if (url.match('#')) {
        if ($('a[href="#' + url.split('#')[1]).length) {
            $('a[href="#' + url.split('#')[1]).tab('show');
        }
    }
       var loadingTask = pdfjsLib.getDocument($('.url_pdf').text());

        var thePdf = null;

    loadingTask.promise.then(function(pdf) {
        thePdf = pdf;

Failure occurs in the getDocument.
Uncaught SyntaxError: Unexpected token .    pdf.js:2370
Line is pdf.js:2370:
File:
2354:     if (!intentState.displayReadyCapability) {
2355:       intentState.displayReadyCapability = (0, _util.createPromiseCapability)();
2356:       intentState.operatorList = {
2357:         fnArray: [],
2358:         argsArray: [],
2359:         lastChunk: false
2360:       };
2361:
2362:       if (this._stats) {
2363:         this._stats.time("Page Request");
2364:       }
2365:
2366:       this._pumpOperatorList({
2367:         pageIndex: this._pageIndex,
2368:         intent: renderingIntent,
2369:         renderInteractiveForms: renderInteractiveForms === true,
2370:         annotationStorage: annotationStorage?.getAll() || null
2371:       });
2372:     }
2373:
2374:     const complete = error => {
2375:       const i = intentState.renderTasks.indexOf(internalRenderTask);
2376:
2377:       if (i >= 0) {
2378:         intentState.renderTasks.splice(i, 1);
2379:       }
2380:
2381:       if (this.cleanupAfterRender || renderingIntent === "print") {
2382:         this.pendingCleanup = true;
2383:       }
2384:
2385:       this._tryCleanup();
seems to fail to load the document in annotationStorage.  My code loads the documents  perfectly for mozilla/chrome and any other browser (even in phone). Failure is related to Mi UI Browser with the version stated before.

annotationStorage: annotationStorage?.getAll() || null

This line uses optional chaining, and it looks like your browser doesn't support that. This means you're using the non-ES5 version of PDF.js in a ES5-only browser, so you should switch to the ES5 build instead.

Never link to this URL directly since it changes on every push to master and may be broken. Either use a CDN (see https://github.com/mozilla/pdf.js#including-via-a-cdn) or host the file yourself.


[DO NOT MERGE] test unit test

Codecov Report

Merging #13335 (bee3609) into master (d7c94b5) will decrease coverage by 21.10%.
The diff coverage is 100.00%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7c94b5...bee3609. Read the comment docs.


chore: improve ci

### Definition of Ready
- [ ] I am happy with the code
- [ ] Short description of the feature/issue is added in the pr description
- [ ] PR is linked to the corresponding user story
- [ ] Acceptance criteria are met
- [ ] All open todos and follow ups are defined in a new ticket and justified
- [ ] Deviations from the acceptance criteria and design are agreed with the PO and documented.
- [ ] No debug or dead code
- [ ] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [ ] Documentation/examples are up-to-date
- [ ] All non-functional requirements are met
- [ ] Functionality of the acceptance criteria is checked manually on the dev system.

Should we also move lint and build to the Makefile? And add a Dockerfile so Windows users and users who don't want to install all build dependencies can also use the Makefile?

replaced by https://github.com/zitadel/zitadel/pull/5953


Fix floating-point comparison in QueryAssertions for aggregation

Summary: assertEqualResults and assertResults in QueryAssertions.cpp compare the actual and expected result vectors through a data structure std::multiset<MaterializedRow, MaterializedRowComparator> where floating-point values are compared with epsilon. This data structure has undefined behavior because MaterializedRowComparator doesn't satisfy the strict weak ordering required by std (https://github.com/facebookincubator/velox/issues/3493). This bug in particular affects the verification of aggregation functions because floating-point results are computed via different query plans and hence the comparison needs to tolerate imprecision. This diff fixes this bug for aggregation by reordering the result columns to put the floating-point column at the end and grouping the results via sorting by the rest columns. If there is only one floating-point column or every group only contains one result row, the result rows are then compared via a merge-join logic with epsilon. Otherwise, the result rows are compared directly without epsilon.
The requirement for every group containing only one result row when there are multiple floating point columns is necessary because the imprecision in one floating point column can make the sorted rows disordered with regards to the other floating point columns.
E.g., Expected rows:

c0
c1

1.0
1.0

1.0001
2.0

Actual rows:

c0
c1

1.0001
2.0

1.001
1.0

Window aggregation test is tricky because there can be multiple result rows for one group but it requires comparison with epsilon. So we add an additional identifier input vector to force each group in the result containing one row only.
Reviewed By: Yuhta
Differential Revision: D42557674

@kagamiori Wei, thank you for working on this. I still need to read the code changes QueryAssertions.cpp, but here are some initial comments on other parts of the PR.

Window aggregation test is tricky because if its input is a floating-point column, result rows grouped by non-floating-point columns won't satisfy the requirement of bucket size being 1. So we project out the input columns after getting the window aggregation result in this situation.

This part of the PR description is not very clear and doesn't seem to match the code changes. Specifically, it is not clear how "we project out the input columns after getting the window aggregation result" helps fix the issue. From the code changes I'm seeing that you are adding an extra "row number" column to the input with unique values of type BIGINT.
It seems that we are introducing a limitation to assertQuery and similar methods, i.e. inputs with floating points columns can be compared only if non-floating point columns form a unique key. It would be nice to document that limitation.

Hi Masha, the PR description was for the last version of the code. I updated it internally but it didn't get synced up.

@kagamiori Wei, thank you for working on this. I still need to read the code changes QueryAssertions.cpp, but here are some initial comments on other parts of the PR.

Window aggregation test is tricky because if its input is a floating-point column, result rows grouped by non-floating-point columns won't satisfy the requirement of bucket size being 1. So we project out the input columns after getting the window aggregation result in this situation.

This part of the PR description is not very clear and doesn't seem to match the code changes. Specifically, it is not clear how "we project out the input columns after getting the window aggregation result" helps fix the issue. From the code changes I'm seeing that you are adding an extra "row number" column to the input with unique values of type BIGINT.
It seems that we are introducing a limitation to assertQuery and similar methods, i.e. inputs with floating points columns can be compared only if non-floating point columns form a unique key. It would be nice to document that limitation.

Hi Masha, the PR description was for the last version of the code. I updated the code and description internally, but the description didn't get synced to GitHub. I've updated it manually and will add comments to the code as well.


kubectl apply -k with helmCharts cannot run

What happened:
Can't pass the kustomize flag --enable-helm to kubectl apply -k
What you expected to happen:
Helm chart gets built and deployed.
How to reproduce it (as minimally and precisely as possible):

create this kustomize file: test/kustomization.yaml

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmCharts:
  - name: minecraft
    releaseName: test
    version: 3.1.3
    repo: https://itzg.github.io/minecraft-server-charts

Try to deploy it with  kubectl.exe apply -k test
This happens:

error: trouble configuring builtin HelmChartInflationGenerator with config: `
name: minecraft
releaseName: test
repo: https://itzg.github.io/minecraft-server-charts
version: 3.1.3
`: must specify --enable-helm

However, I found no way to pass --enable-helm
Anything else we need to know?:
using kustomize build --enable-helm test (version v4.1.3) works as expected.
The doc about helmCharts is very minimal. is there some way to enable kustomize flags globally from kubectl?
Environment:

Kubernetes client and server versions (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.0", GitCommit:"c2b5237ccd9c0f1d600d3072634ca66cefdf272f", GitTreeState:"clean", BuildDate:"2021-08-04T18:03:20Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"windows/amd64"} Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.4", GitCommit:"3cce4a82b44f032d0cd1a1790e6d2f5a55d20aae", GitTreeState:"clean", BuildDate:"2021-08-11T18:10:22Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}
Cloud provider or hardware configuration: docker desktop k8s on WSL
OS (e.g: cat /etc/os-release): windows 10

Have the same question.
Is it a new feature? Cause someone definitely tried it before 😏.

You're right that kubectl apply -k does not expose additional Kustomize-specific flags. I believe it intentionally only supports the simple case, for the sake of the simplicity of apply options. If you need to configure Kustomize behavior with additional flags, you can use kubectl kustomize and standalone kustomize instead.
/close

Thanks, I managed to get it to work with the following:
kubectl kustomize test --enable-helm | kubectl apply -f -

@KnVerey the proposed usage with kubectl apply -f - would work, but that would not be enough with ArgoCD ...which is my case

I'm not familiar enough with ArgoC to advise, though it appears to me that it supports arbitrary Kustomize versions, which would imply standalone Kustomize support. I maintain that kubectl apply -k must not be modified to support Kustomize-specific flags.

@yossicohn argocd supports this via an option in the argocd CM.
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  # enables a kustomization option that can work with remote helm repos
  kustomize.buildOptions: --enable-helm

@zswanson thanks a lot


What if I rely on Nokogiri from Bundler?

Cleaning up a bit of lingering CI weirdness from #741.

🚀 Deployed on https://661752d41d4687243b77cd1d--alexwlchan.netlify.app


Update detection of EA Launcher cloud saves

https://www.pcgamingwiki.com/wiki/Template:Save_game_cloud_syncing
Currently only checking for origin

This should be fixed too, yes


[ENH] preserve index.freq in get_cutoff

This PR ensures that the freq attribute is preserved if get_cutoff returns an index.
Also simplifies the return_index case distinctions by refactoring it into a function sub_index, and fixes an unreported bug for pd-multiindex where the returned index would always be the last, even if reverse_order=True.

Please, add unit tests. Especially since you are claiming that this PR fixes some unreported bugs. These tests should fail on master. Thanks.

Here you go, some freshly cooked tests coming up 😄


BUG Using ports different than 80/443 within alternate_base_url

Currently, specifying a URL with a port number in alternate_base_url different than 80 (http) or 443 (https), will make the Director class not to detect this and break most URLs in the application.
Note that in our case, the URL must be specified in most cases when SilverStripe is working on a URL prefix, e.g. /silverstripe.
Steps to reproduce:

Change the frontend's HTTP port to something different than 80.
Set alternate_base_url for SilverStripe to use this new port:

SilverStripe\Control\Director:
  alternate_base_url: http://HOSTNAME:PORT/silverstripe

Next, if you access SilverStripe, you will see that most URLs are wrong and stylesheets are not loading.
For instance, by setting the URL http://<IP_ADDRESS>:8080/silverstripe, the <base> tag appears as following:
<base href="http://<IP_ADDRESS>/silverstripe/"><!--[if lte IE 6]></base><![endif]-->

And no mentions of the port 8080.

It seems reasonable to support this IMO. My initial thought is that perhaps the port should be set/resolved separately by a new method (e.g. Director::port()). Director::host() is designed to return the hostname, and the port number isn’t really part of that...
I’ll await input from those who deal with this class more often than I do before suggesting you make that change ;)

Yes it seems to be a real problem:

Add Director::port()
Ensure Director::protocolAndHost() includes port. (seems to be more correct than omitting in > 50% of cases). This will ensure that Director::absoluteURL respects port.
Ensure Director::absoluteBaseURLWithAuth() includes port.

To respond to this direct PR; I'm not sure we can safely include the port number in host() method return. It would break code which relied on the pure-hostname being returned from this as a low-level method.

Hi, thanks for checking, I'll try to find some time in the next couple of days to look into improving this.