Building using native Visual Studio

Describe the bug
Not a bug. I am just wondering what prevents building filament using the native Microsoft compiler in Visual Studio? Is it just a matter of C++ code not building?
I saw a reference about assembly code in another issue. Is this still a problem or is there equivalent c code?

msvc encounters a lot of errors and warnings when attempting to build our codebase. We are not willing to implement some of the fixes required to make our code compile with msvc. That said we haven't tried with VS2019 which might behave better (unfortunately our CI is limited to VS2015 at the moment.
Note that you can use a prebuilt version of Filament with msvc; our public headers should work just fine, if they don't please let us know.

Thanks a lot for the quick answer. I just need to say that filament is awesome, and the doc too!
Unfortunately, The public headers don't quite work with msvc.
vs2015 was having lots of issues with constexpr, so I switched to vs2019, but even then the headers have issues.
For example in utils/compilers.h we hit the #error Compiler does not support the packed attribute.. This #error can be commented out as the macro UTILS_PACKED doesn't seem to be referenced by the other headers.
Another issue which shows up when building a sample (however this may not be required for using filament) is the following on vec2, vec3, vec4:
error C2976:  'filament::math::vec2': too few template arguments on:
using half2 = vec2;
Although half is defined as is_floating_point in half.h, it is not defined as is_arithmetic which is what is tested in:
template <typename T, typename = typename std::enable_if<std::is_arithmetic<T>::value >::type>
using vec2 = details::TVec2<T>;

Possibly libc++ automatically does that, but not msvc. In any case I'm not sure how useful the std::enable_if in the vec2, vec3 and vec4 definitions is (besides giving a shrter error is someone tries to define a vec2 of a non-arithmetic type).
I'll keep trying to build the material_sandbox sample using native vs2019.

Thanks for the info, we should fix the headers.
@bejado Ben, do you mind fixing those issue? The problem in half.h should be easy to fix.

Yes, just adding in half.h:
#if defined(_MSC_VER)
    template<> struct is_arithmetic< filament::math::half> : public std::true_type {};
#endif

is enough. But if you guys want, give me a couple days and I'll send you a PR.

We don't need to make is msvc specific right? I don't see any reason why we couldn't do this with clang.

I was just concerned that if there is the same definition in libc++ it might cause an error, but I just tried rebuilding with clang on my pc with the non-msvc specific change and it built fine.

Another issue is this:
1>C:\greg\filament\samples - Copy\app\CameraManipulator.cpp(74,29): error C2666:  'filament::math::details::TQuatProductOperators<filament::math::details::TQuaternion,T>::operator *': 13 overloads have similar conversions
1>C:\greg\filament\samples - Copy\app\CameraManipulator.cpp(74,29): error C2666:         with
1>C:\greg\filament\samples - Copy\app\CameraManipulator.cpp(74,29): error C2666:         [
1>C:\greg\filament\samples - Copy\app\CameraManipulator.cpp(74,29): error C2666:             T=double
1>C:\greg\filament\samples - Copy\app\CameraManipulator.cpp(74,29): error C2666:         ]
1>C:\greg\filament\include\math\TQuatHelpers.h(130,39): message :  could be 'filament::math::details::TQuaternion<double> filament::math::details::TQuatProductOperators<filament::math::details::TQuaternion,T>::operator *(filament::math::details::TQuaternion<T>,T)' [found using argument-dependent lookup]
1>C:\greg\filament\include\math\TQuatHelpers.h(130,39): message :         with
1>C:\greg\filament\include\math\TQuatHelpers.h(130,39): message :         [
1>C:\greg\filament\include\math\TQuatHelpers.h(130,39): message :             T=double
1>C:\greg\filament\include\math\TQuatHelpers.h(130,39): message :         ]
1>C:\greg\filament\include\math\TQuatHelpers.h(135,39): message :  or       'filament::math::details::TQuaternion<double> filament::math::details::TQuatProductOperators<filament::math::details::TQuaternion,T>::operator *(T,filament::math::details::TQuaternion<T>)' [found using argument-dependent lookup]
1>C:\greg\filament\include\math\TQuatHelpers.h(135,39): message :         with
1>C:\greg\filament\include\math\TQuatHelpers.h(135,39): message :         [
1>C:\greg\filament\include\math\TQuatHelpers.h(135,39): message :             T=double
1>C:\greg\filament\include\math\TQuatHelpers.h(135,39): message :         ]
1>C:\greg\filament\include\math\TVecHelpers.h(204,49): message :  or       'filament::math::details::TVec4<double> filament::math::details::TVecProductOperators<filament::math::details::TVec4,T>::operator *(filament::math::details::TVec4<T>,const filament::math::details::TVec4<T> &)' [found using argument-dependent lookup]
1>C:\greg\filament\include\math\TVecHelpers.h(204,49): message :         with
1>C:\greg\filament\include\math\TVecHelpers.h(204,49): message :         [
1>C:\greg\filament\include\math\TVecHelpers.h(204,49): message :             T=double
1>C:\greg\filament\include\math\TVecHelpers.h(204,49): message :         ]
...

I guess this is because the quaternions and vec4 and vec3 ... can all be implicitely constructed with a float or a double, because the following are not explicit. I wonder how this is not an issue with clang:
    // handles implicit conversion to a tvec4. must not be explicit.
    template<typename A>
    constexpr TQuaternion(A w) : x(0), y(0), z(0), w(w) {
        static_assert(std::is_arithmetic<A>::value, "requires arithmetic type");
    }

and
    // handles implicit conversion to a tvec4. must not be explicit.
    template<typename A>
    constexpr TVec4(A v) : x(v), y(v), z(v), w(v) { }

The same issue happens with vec2/3/4. myVec2 + 0.5 yields an error with MSVC (at least with VS2015). And it's one of ergonomics we don't want to compromise.

Well, we really should figure out which compiler is correct, because if clang is not standard conformant by allowing this, it could change in a future release.
I wonder how it is possible to feel confident in what the code actually compiles to with all these implicit conversions?

gcc was fine with that code too, but yeah I agree.

@pixelflinger who owns this code.

Some hints can probably be found here: https://en.cppreference.com/w/cpp/language/implicit_conversion

Yes. I'll try reproducing the issue in a small program so it is easier to figure out which compiler is correct.

Confirmed as still an issue on vs2019.
Also confirmed adding is_arithmentic fixes it,
template<> struct is_arithmetic< filament::math::half> : public std::true_type {};

C:\projects\XXX\filament-20190426-windows\include\math\vec2.h(102): error C2976: 'filament::math::vec2': too few template arguments
  C:\projects\XXX\filament-20190426-windows\include\math/vec2.h(98): note: see declaration of 'filament::math::vec2'
C:\projects\XXX\filament-20190426-windows\include\math\vec3.h(121): error C2976: 'filament::math::vec3': too few template arguments
  C:\projects\XXX\filament-20190426-windows\include\math/vec3.h(117): note: see declaration of 'filament::math::vec3'
C:\projects\XXX\filament-20190426-windows\include\math\vec4.h(113): error C2976: 'filament::math::vec4': too few template arguments
  C:\projects\XXX\filament-20190426-windows\include\math/vec4.h(109): note: see declaration of 'filament::math::vec4'

It may be more involved than I originally thought to reproduce the error C2666:  'filament::math::details::TQuatProductOperators<filament::math::details::TQuaternion,T>::operator *': 13 overloads have similar conversions  issue. I tried the code below, but it does build fine with msvc as well as gcc.
#include <cstdio>
#include <cassert>

#if __has_attribute(pure)
#   define MATH_PURE __attribute__((pure))
#else
#   define MATH_PURE
#endif

#ifdef _MSC_VER
    #define MATH_EMPTY_BASES __declspec(empty_bases)
    #define MATH_NOUNROLL
#else // _MSC_VER
    #define MATH_EMPTY_BASES
    #define MATH_NOUNROLL _Pragma("nounroll")
#endif // _MSC_VER

template <template<typename T> class QUATERNION, typename T>
class TQuatProductOperators {
public:
    constexpr QUATERNION<T>& operator *= (T v) {
        QUATERNION<T>& lhs = static_cast<QUATERNION<T>&>(*this);
        for (size_t i = 0; i < QUATERNION<T>::size(); i++) {
            lhs[i] *= v;
        }
        return lhs;
    }

    friend inline
    constexpr QUATERNION<T> MATH_PURE operator *(QUATERNION<T> q, T scalar) {
        // don't pass q by reference because we need a copy anyways
        return q *= scalar;
    }
    friend inline
    constexpr QUATERNION<T> MATH_PURE operator *(T scalar, QUATERNION<T> q) {
        // don't pass q by reference because we need a copy anyways
        return q *= scalar;
    }

    friend inline
    constexpr QUATERNION<T> MATH_PURE operator /(QUATERNION<T> q, T scalar) {
        // don't pass q by reference because we need a copy anyways
        return q /= scalar;
    }
};

template <typename T>
class MATH_EMPTY_BASES TQuaternion : public TQuatProductOperators<TQuaternion, T>
{

public:
    enum { SIZE = 4 };
    inline constexpr static size_t size() { return SIZE; }

    inline constexpr T& operator[](size_t i) {
        assert(i < SIZE);
        return (&x)[i];
    }

    template<typename A>
    constexpr TQuaternion(A w) : x(0), y(0), z(0), w(w) {
    }

    union {
        struct { T x, y, z, w; };
    };

};

template<template<typename T> class VECTOR, typename T>
class TVecProductOperators {
public:
    /* compound assignment from a another vector of the same size but different
     * element type.
     */
    template<typename OTHER>
    constexpr VECTOR<T>& operator *=(const VECTOR<OTHER>& v) {
        VECTOR<T>& lhs = static_cast<VECTOR<T>&>(*this);
        for (size_t i = 0; i < lhs.size(); i++) {
            lhs[i] *= v[i];
        }
        return lhs;
    }

    /*
     * NOTE: the functions below ARE NOT member methods. They are friend functions
     * with they definition inlined with their declaration. This makes these
     * template functions available to the compiler when (and only when) this class
     * is instantiated, at which point they're only templated on the 2nd parameter
     * (the first one, BASE<T> being known).
     */

    /* The operators below handle operation between vectors of the same size
     * but of a different element type.
     */
    template<typename RT>
    friend inline constexpr VECTOR<T> MATH_PURE operator *(VECTOR<T> lv, const VECTOR<RT>& rv) {
        // don't pass lv by reference because we need a copy anyways
        return lv *= rv;
    }

    /* The operators below (which are not templates once this class is instanced,
     * i.e.: BASE<T> is known) can be used for implicit conversion on both sides.
     * These handle operations like "vector * scalar" and "scalar * vector" by
     * letting the compiler implicitly convert a scalar to a vector (assuming
     * the BASE<T> allows it).
     */
    friend inline constexpr VECTOR<T> MATH_PURE operator *(VECTOR<T> lv, const VECTOR<T>& rv) {
        // don't pass lv by reference because we need a copy anyways
        return lv *= rv;
    }
};

template <typename T>
class MATH_EMPTY_BASES TVec3 : public TVecProductOperators<TVec3, T> {
public:
    static constexpr size_t SIZE = 3;

    union {
        T v[SIZE];
        struct { T x, y, z; };
    };

    inline constexpr size_t size() const { return SIZE; }

    // array access
    inline constexpr T const& operator[](size_t i) const {
        // only possible in C++0x14 with constexpr
        assert(i < SIZE);
        return v[i];
    }

    inline constexpr T& operator[](size_t i) {
        assert(i < SIZE);
        return v[i];
    }

    // constructors

    // default constructor
    constexpr TVec3() = default;

    // handles implicit conversion to a tvec4. must not be explicit.
    template<typename A>
    constexpr TVec3(A v) : x(v), y(v), z(v) { }

    template<typename A, typename B, typename C>
    constexpr TVec3(A x, B y, C z) : x(x), y(y), z(z) { }
};

template <typename T> using vec3 = TVec3<T>;

template <typename T>
class MATH_EMPTY_BASES TVec4 :  public TVecProductOperators<TVec4, T>{
public:
    typedef T value_type;
    typedef T& reference;
    typedef T const& const_reference;
    typedef size_t size_type;
    static constexpr size_t SIZE = 4;

    union {
        T v[SIZE];
        struct { T x, y, z, w; };
    };

    inline constexpr size_t size() const { return SIZE; }

    // array access
    inline constexpr T const& operator[](size_t i) const {
        // only possible in C++0x14 with constexpr
        assert(i < SIZE);
        return v[i];
    }

    inline constexpr T& operator[](size_t i) {
        assert(i < SIZE);
        return v[i];
    }

    // constructors

    // default constructor
    constexpr TVec4() = default;

    // handles implicit conversion to a tvec4. must not be explicit.
    template<typename A>
    constexpr TVec4(A v) : x(v), y(v), z(v), w(v) { }

    template<typename A, typename B, typename C, typename D>
    constexpr TVec4(A x, B y, C z, D w) : x(x), y(y), z(z), w(w) { }

    template<typename A, typename B>
    constexpr TVec4(const TVec3<A>& v, B w) : x(v.x), y(v.y), z(v.z), w(w) { }

    template<typename A>
    constexpr TVec4(const TVec4<A>& v) : x(v.x), y(v.y), z(v.z), w(v.w) { }
};

using double3 = vec3<double>;

int main()
{
    double3 d_s = { 1.0, 0.0, 0.0};
    double3 d_t = { 0.0, 1.0, 0.0};

    double s = 1;
    double t = 2;

    double3 mTranslation = (s * d_s);
    double3 tTranslation = (t * d_t);
    return (int)mTranslation[0];
}

@meshula Thanks for the update!
@gpyalt I think we'll need to wait for @pixelflinger to chime in :/

Thanks @romainguy  and @meshula!

One more note, I'm attempting to use msvc to link, but running into issues vs opengl32.lib. I added some notes on this related issue: https://github.com/google/filament/issues/652#issuecomment-500646203

@bejado To solve our CI problem maybe we could setup a CircleCI or similar for the Windows build? Releases will be a bit more annoying but we'd have access to more recent compilers I think.

And VS Community is available for Open Source projects so that could work for us.

I think that is_arithmetic should be true for any type that is_floating_point. This is not technically the C++ spec, but this is what it says on cppreference.com:
If T is an arithmetic type (that is, an integral type or a floating-point type) or a cv-qualified version thereof, provides the member constant value equal true. For any other type, value is false.

So this looks like an MSVC issue. I'll have a workaround in filament.

Clang 8.0.1 on Windows has the same problem with is_arithmetic. clang-cl unable to compile the code with original headers:

is_arithmetic logs
In file included from ..\source\filament.cpp:2:
In file included from ..\source/filament.h:52:
In file included from ..\include\filament/Engine.h:20:
In file included from ..\include\filament/Camera.h:22:
In file included from ..\include\filament/Frustum.h:22:
In file included from ..\include\filament/Box.h:26:
In file included from ..\include\math/mat4.h:20:
In file included from ..\include\math/mat3.h:20:
In file included from ..\include\math/quat.h:21:
In file included from ..\include\math/TQuatHelpers.h:28:
In file included from ..\include\math/vec3.h:20:
..\include\math/vec2.h(97,58): error: failed requirement 'std::is_arithmetic<filament::math::half>::value'; 'enable_if'
      cannot be used to disable this declaration
template <typename T, typename = typename std::enable_if<std::is_arithmetic<T>::value >::type>
                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
..\include\math/vec2.h(102,15): note: in instantiation of default argument for 'vec2<filament::math::half>' required
      here
using half2 = vec2<half>;
              ^~~~~~~~~~
In file included from ..\source\filament.cpp:2:
In file included from ..\source/filament.h:52:
In file included from ..\include\filament/Engine.h:20:
In file included from ..\include\filament/Camera.h:22:
In file included from ..\include\filament/Frustum.h:22:
In file included from ..\include\filament/Box.h:26:
In file included from ..\include\math/mat4.h:20:
In file included from ..\include\math/mat3.h:20:
In file included from ..\include\math/quat.h:21:
In file included from ..\include\math/TQuatHelpers.h:28:
..\include\math/vec3.h(116,58): error: failed requirement 'std::is_arithmetic<filament::math::half>::value'; 'enable_if'
      cannot be used to disable this declaration
template <typename T, typename = typename std::enable_if<std::is_arithmetic<T>::value >::type>
                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
..\include\math/vec3.h(121,15): note: in instantiation of default argument for 'vec3<filament::math::half>' required
      here
using half3 = vec3<half>;
              ^~~~~~~~~~
In file included from ..\source\filament.cpp:2:
In file included from ..\source/filament.h:52:
In file included from ..\include\filament/Engine.h:20:
In file included from ..\include\filament/Camera.h:22:
In file included from ..\include\filament/Frustum.h:22:
In file included from ..\include\filament/Box.h:26:
In file included from ..\include\math/mat4.h:20:
In file included from ..\include\math/mat3.h:20:
In file included from ..\include\math/quat.h:23:
..\include\math/vec4.h(108,58): error: failed requirement 'std::is_arithmetic<filament::math::half>::value'; 'enable_if'
      cannot be used to disable this declaration
template <typename T, typename = typename std::enable_if<std::is_arithmetic<T>::value >::type>
                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
..\include\math/vec4.h(113,15): note: in instantiation of default argument for 'vec4<filament::math::half>' required
      here
using half4 = vec4<half>;

There's also a few other errors:

M_PI and length2 logs
In file included from ..\source\filament.cpp:2:
In file included from ..\source/filament.h:52:
In file included from ..\include\filament/Engine.h:20:
In file included from ..\include\filament/Camera.h:22:
In file included from ..\include\filament/Frustum.h:22:
In file included from ..\include\filament/Box.h:26:
..\include\math/mat4.h(495,28): error: use of undeclared identifier 'M_PI'; unqualified lookup into dependent bases of
      class template 'TMat44' is a Microsoft extension [-Werror,-Wmicrosoft-template]
        h = std::tan(fov * M_PI / 360.0f) * near;
                           ^
..\include\math/mat4.h(498,28): error: use of undeclared identifier 'M_PI'; unqualified lookup into dependent bases of
      class template 'TMat44' is a Microsoft extension [-Werror,-Wmicrosoft-template]
        w = std::tan(fov * M_PI / 360.0f) * near;
                           ^
In file included from ..\source\filament.cpp:2:
In file included from ..\source/filament.h:52:
In file included from ..\include\filament/Engine.h:20:
In file included from ..\include\filament/Camera.h:22:
In file included from ..\include\filament/Frustum.h:22:
..\include\filament/Box.h(47,36): error: comparing floating point with == or != is unsafe [-Werror,-Wfloat-equal]
        return length2(halfExtent) == 0;
               ~~~~~~~~~~~~~~~~~~~ ^  ~
6 errors generated.

MSVC 19.22.27905 small warning:
filament.cpp
..\include\utils/algorithm.h(49): warning C4293: '>>': shift count negative or too big, undefined behavior
..\include\utils/algorithm.h(77): note: see reference to function template instantiation 'T utils::details::clz<unsigned int>(T) noexcept' being compiled
        with
        [
            T=unsigned int
        ]

Hi @nxrighthere, I am still working on building all filament with vs2019. At this point I am not concerned with the warnings, just trying to get it all to build. The main issue I have is with aggregate initializers, where filament code uses some non-standard features (that clang supports). Sometimes the fix is easy, like use the struct order even when .designators are used. I'm making good progress, and hopefully I'll be done in a few days and have a PR to submit to the filament team.

@gpyalt Do you have examples of non-standard features Filament relies on? It's definitely not our intent to use features that are not part of the C++ standard.

@romainguy, sure, I am glad to hear that! (ps: this is my personal github acct - same as @gpyalt)
As I said, I plan to send a PR with a bunch of diffs. Or I can send them piecemeal so there are easier to digest. Just for info, whatever you prefer, I also have changes in the CMakeLists.txt so that the visual studio config works.
Here is some things I found
examples:

aggregate initialization (https://en.cppreference.com/w/cpp/language/aggregate_initialization)
Note: out-of-order designated initialization, nested designated initialization, mixing of designated initializers and regular initializers, and designated initialization of arrays are all supported in the C programming language, but are not allowed in C++.
struct B b = {.a.x = 0};       // valid C, invalid C++ (nested)

nested: multiple instances. For example in PostProcessManager.cpp, FrameGraph.cpp
{ .attachments.color = { data.ssao, FrameGraphRenderTarget::Attachments::WRITE },
  .attachments.depth = { data.depth, FrameGraphRenderTarget::Attachments::READ }

example of change:

out of order:  multiple instances (Renderer.cpp, ...)
data.depth = builder.createTexture("Depth Buffer", {
                            .width = svp.width, .height = svp.height,
                            .samples = msaa,
                            .format = TextureFormat::DEPTH24
                    });

example of change:

union initialization struct FroxelEntry. Forgot what the exact error is. My change:

use of clang specific stuff. example of change:

Thanks, that's useful.

Please don't make any change yet... I'll be submitting pull requests soon.

I basically have everything building with vs2019/msvc, last issue is some undefined symbols when linking suzanne (related to backend/bluegl):
1>------ Build started: Project: suzanne, Configuration: Debug x64 ------
1>LINK : warning LNK4044: unrecognized option '/Wl,--gc-sections'; ignored
1>backend.lib(OpenGLDriver.obj) : error LNK2019: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64,unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::beginFrame(__int64,unsigned int)>::log(void)" (?log@?$Command@$1?beginFrame@Driver@backend@filament@@QEAAX_JI@Z@?$CommandType@P8Driver@backend@filament@@EAAX_JI@Z@backend@filament@@AEAAXXZ) referenced in function "public: static void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64,unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::beginFrame(__int64,unsigned int)>::execute<void (__cdecl filament::OpenGLDriver::*)(__int64,unsigned int),class filament::OpenGLDriver &>(void (__cdecl filament::OpenGLDriver::*&&)(__int64,unsigned int),class filament::OpenGLDriver &,class filament::backend::CommandBase *,__int64 *)" (??$execute@P8OpenGLDriver@filament@@EAAX_JI@ZAEAV12@@?$Command@$1?beginFrame@Driver@backend@filament@@QEAAX_JI@Z@?$CommandType@P8Driver@backend@filament@@EAAX_JI@Z@backend@filament@@SAX$$QEAP8OpenGLDriver@3@EAAX_JI@ZAEAV43@PEAVCommandBase@23@PEA_J@Z)
1>backend.lib(NoopDriver.obj) : error LNK2001: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64,unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::beginFrame(__int64,unsigned int)>::log(void)" (?log@?$Command@$1?beginFrame@Driver@backend@filament@@QEAAX_JI@Z@?$CommandType@P8Driver@backend@filament@@EAAX_JI@Z@backend@filament@@AEAAXXZ)
1>backend.lib(OpenGLDriver.obj) : error LNK2019: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64)>::Command<&public: void __cdecl filament::backend::Driver::setPresentationTime(__int64)>::log(void)" (?log@?$Command@$1?setPresentationTime@Driver@backend@filament@@QEAAX_J@Z@?$CommandType@P8Driver@backend@filament@@EAAX_J@Z@backend@filament@@AEAAXXZ) referenced in function "public: static void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64)>::Command<&public: void __cdecl filament::backend::Driver::setPresentationTime(__int64)>::execute<void (__cdecl filament::OpenGLDriver::*)(__int64),class filament::OpenGLDriver &>(void (__cdecl filament::OpenGLDriver::*&&)(__int64),class filament::OpenGLDriver &,class filament::backend::CommandBase *,__int64 *)" (??$execute@P8OpenGLDriver@filament@@EAAX_J@ZAEAV12@@?$Command@$1?setPresentationTime@Driver@backend@filament@@QEAAX_J@Z@?$CommandType@P8Driver@backend@filament@@EAAX_J@Z@backend@filament@@SAX$$QEAP8OpenGLDriver@3@EAAX_J@ZAEAV43@PEAVCommandBase@23@PEA_J@Z)
1>backend.lib(NoopDriver.obj) : error LNK2001: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64)>::Command<&public: void __cdecl filament::backend::Driver::setPresentationTime(__int64)>::log(void)" (?log@?$Command@$1?setPresentationTime@Driver@backend@filament@@QEAAX_J@Z@?$CommandType@P8Driver@backend@filament@@EAAX_J@Z@backend@filament@@AEAAXXZ)
1>backend.lib(OpenGLDriver.obj) : error LNK2019: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::endFrame(unsigned int)>::log(void)" (?log@?$Command@$1?endFrame@Driver@backend@filament@@QEAAXI@Z@?$CommandType@P8Driver@backend@filament@@EAAXI@Z@backend@filament@@AEAAXXZ) referenced in function "public: static void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::endFrame(unsigned int)>::execute<void (__cdecl filament::OpenGLDriver::*)(unsigned int),class filament::OpenGLDriver &>(void (__cdecl filament::OpenGLDriver::*&&)(unsigned int),class filament::OpenGLDriver &,class filament::backend::CommandBase *,__int64 *)" (??$execute@P8OpenGLDriver@filament@@EAAXI@ZAEAV12@@?$Command@$1?endFrame@Driver@backend@filament@@QEAAXI@Z@?$CommandType@P8Driver@backend@filament@@EAAXI@Z@backend@filament@@SAX$$QEAP8OpenGLDriver@3@EAAXI@ZAEAV43@PEAVCommandBase@23@PEA_J@Z)
1>backend.lib(NoopDriver.obj) : error LNK2001: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(unsigned int)>::Command<&public: void __cdecl filament::backend::Driver::endFrame(unsigned int)>::log(void)" (?log@?$Command@$1?endFrame@Driver@backend@filament@@QEAAXI@Z@?$CommandType@P8Driver@backend@filament@@EAAXI@Z@backend@filament@@AEAAXXZ)
...
1>backend.lib(OpenGLDriver.obj) : error LNK2019: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::Command<&public: void __cdecl filament::backend::Driver::draw(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::log(void)" (?log@?$Command@$1?draw@Driver@backend@filament@@QEAAXUPipelineState@34@U?$Handle@UHwRenderPrimitive@backend@filament@@@34@@Z@?$CommandType@P8Driver@backend@filament@@EAAXUPipelineState@23@U?$Handle@UHwRenderPrimitive@backend@filament@@@23@@Z@backend@filament@@AEAAXXZ) referenced in function "public: static void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::Command<&public: void __cdecl filament::backend::Driver::draw(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::execute<void (__cdecl filament::OpenGLDriver::*)(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>),class filament::OpenGLDriver &>(void (__cdecl filament::OpenGLDriver::*&&)(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>),class filament::OpenGLDriver &,class filament::backend::CommandBase *,__int64 *)" (??$execute@P8OpenGLDriver@filament@@EAAXUPipelineState@backend@2@U?$Handle@UHwRenderPrimitive@backend@filament@@@42@@ZAEAV12@@?$Command@$1?draw@Driver@backend@filament@@QEAAXUPipelineState@34@U?$Handle@UHwRenderPrimitive@backend@filament@@@34@@Z@?$CommandType@P8Driver@backend@filament@@EAAXUPipelineState@23@U?$Handle@UHwRenderPrimitive@backend@filament@@@23@@Z@backend@filament@@SAX$$QEAP8OpenGLDriver@3@EAAXUPipelineState@23@U?$Handle@UHwRenderPrimitive@backend@filament@@@23@@ZAEAV43@PEAVCommandBase@23@PEA_J@Z)
1>backend.lib(NoopDriver.obj) : error LNK2001: unresolved external symbol "private: void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::Command<&public: void __cdecl filament::backend::Driver::draw(struct filament::backend::PipelineState,struct filament::backend::Handle<struct filament::backend::HwRenderPrimitive>)>::log(void)" (?log@?$Command@$1?draw@Driver@backend@filament@@QEAAXUPipelineState@34@U?$Handle@UHwRenderPrimitive@backend@filament@@@34@@Z@?$CommandType@P8Driver@backend@filament@@EAAXUPipelineState@23@U?$Handle@UHwRenderPrimitive@backend@filament@@@23@@Z@backend@filament@@AEAAXXZ)
1>C:\greg\github\filament\out2\samples\Debug\suzanne.exe : fatal error LNK1120: 53 unresolved externals

Any idea?

bluegl doesn't seem to be involved here. It looks like this method wasn't compiled/exported properly?
template<typename... ARGS>
template<void (Driver::*METHOD)(ARGS...)>
void CommandType<void (Driver::*)(ARGS...)>::Command<METHOD>::log() noexcept  {
    log(std::make_index_sequence<std::tuple_size<SavedParameters>::value>{});
}

It is inside CommandStream.cpp, I wonder if it needs to move somewhere else.

Thanks for the pointer, @romainguy , I'll look into it further.

@romainguy I see the CommandType  template definition, but I don't understand where all the specializations are defined. It seems that it should be with the #include "private/backend/DriverAPI.inc", but this is included only when #if DEBUG_COMMAND_STREAM

@pixelflinger

Undefined symbol (more legible):
void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64,unsigned int)>::
                                Command<&public: void __cdecl filament::backend::Driver::beginFrame(__int64,unsigned int)>::
                                log(void)"

              referenced in function

void __cdecl filament::backend::CommandType<void (__cdecl filament::backend::Driver::*)(__int64,unsigned int)>::
                                Command<&public: void __cdecl filament::backend::Driver::beginFrame(__int64,unsigned int)>::
                                execute<void (__cdecl filament::OpenGLDriver::*)(__int64,unsigned int),class filament::OpenGLDriver &>(void (__cdecl filament::OpenGLDriver::*&&)(__int64,unsigned int),class filament::OpenGLDriver &,class filament::backend::CommandBase *,__int64 *)"

OK, I got it to link.  Indeed the issue was that the CommandType::Command::log() template functions were defined in CommandStream,cpp, but not called from that file. It is called from other cpp files, however none of which had access to the function definition) so MSVC did not instantiate this function in any .o file.
Moving the definition of the two log() functions inline in the  CommandType::Command definition fixed the issue.

Finally, I got the material_sandbox to run on windows, all built with vs2019. I'll keep sending PRs for the changes.

it builds with msvc now, closing the issue :-)


Sketch App now compatible with Version 40

Updates

Plugin should now work with V40. Click to download Zip  and reinstall.

Now allows single integer values. 20, 20:20, 20:20:20, 20:20:10:10. Defaults to 20. See demo.

Now <cmd+j> works only on a text layer and will ignore all other layers. If you select any other layer and hit <cmd+j> It will ignore that layer. No more unwanted folders on unwanted layers.

Demo

It's a very useful plugin. Thank you for working on this.

@ChrissyChien Thanks, Glad you like it.

What about instances where I may have a shape layer based icon next to a text layer and group in a folder. In the past, I could simply commit a padding ratio like 8:16 and Dynamic button would apply a shape layer around the 8:16 name folder. Would really like to see this functionality back, as all button instances are not text-only. Thanks!

Apologies for ignorance on my part. I added a check recently to apply to only the text layer so the user didn't have to undo every time. I'll remove the check and update soon.

Appreciate you making the change! Thanks

Did this previously work with icons? Say you're crating a drop down button. I thought it was possible to also apply the padding rules to the icon layer and it too would update along with the button label. Is this not true?


Create Null Hotkey?

If it's in the docs I didn't spot it, is there a way to quickly create and connect a null to an op?

Hey there is already in stock TD: Alt+N
I have it bound to my mouse personally :)

Mind blown


proposal: tracing certain slow tx execution

Proposal Details
mainnet: 198 & 984
hang nothing has been done, no disk io, but high secp overhead
approach:

add cmd rooch db get-changeset-by-order
re-execute DA chunks and checkout a branch which will save changeset when rooch da exec
searchi changeset by 1

#3031

may caused by bad logic of rooch da exec, because we have to wait long time to load segments from disk, pack into chunk(need verify signature in this process), that's why we see so many secp releated functions.
I've update codes about it, using binary search to find good begin chunk, saving lots of time to reduce the init process.
But I remember tx_order 198 stall is in the middle of the whole execution process, may caused by too big chunk verification signature? and 198 is in this middle of first chunk, there is no reason wait for verification of signature.
Anyway, wait to reproduce it by re-execute from tx_order 1 and save changeset.

not in hurry, move it TBD

after updates da exec logic, no slow tx execution found. To be more precise, the two cases here, 198 and 984, both executed in milliseconds.


Issues with swift 5.2

Just compiled it with 5.2, seems to have some leaks, dangling pointers, etc. I saw from your commit that you have ironed those out. Could you release the package soon?

+1. When can we publish a new version containing this https://github.com/weichsel/ZIPFoundation/pull/159?

Released 🎉

Great thanks!


JavaScript Error Occurred In The Main Process

[x] I have read the contribution documentation for this project.
[x] I agree to follow the code of conduct that this project follows, as appropriate.
[x] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Please note that the package.json file is in my source directory.  The issue was reported by a user when trying to launch our application on Mac OS X.  When I try to launch the application on Mac (my development machine), I don't run into any errors.
Which version of electron-packager are you using?
9.1.0
What cli arguments are you passing?
electron-packager . --overwrite --platform=darwin --arch=x64 --icon=icon.icns --prune=true --out=release-builds
What platform are you running electron-packager on? What platform(s) are you building for?
Running on Mac OS X, building for Mac OS X
Is there a stack trace in the error message you're seeing?
Error: Unable to find a valid app
at Object. (/Applications/TestApp.app/Contents/Resources/electron.asar/browser/init.js:121:9)
When possible, please provide instructions to reproduce your problem:
The issue occurs for the user when trying to launch the application.  The build process completes successfully.
Here's what my package.json file looks like:
{
  "name": "TestApp",
  "version": "2.0.0",
  "description": "Test",
  "main": "main.js",
  "scripts": {
    "start": "electron ."
  },
  "author": "FJA",
  "devDependencies": {
    "electron": "^1.7.9",
    "electron-installer-dmg": "^0.2.1",
    "electron-packager": "^9.1.0"
  }
}

Thanks for filing an issue! In order to help you with your problem, we're going to need more information about it. In particular:

Please provide either a failing minimal testcase (with code) or detailed steps to reproduce your
problem. Using electron-quick-start is a good starting point.

To reproduce the problem, I built the application using the CLI arguments provided above.  I then used electron-installer-dmg to create an installer for the application.  The user then double clicked the DMG, copied the .app to the Applications folder on their Mac and double clicked the .app file to launch it.  Upon trying to launch the application, that's when the error provided occurred for the user.
However, as mentioned, when I follow these steps, the error does not occur for me.  If you would prefer to have me have the user post a comment here about their exact steps that triggered the error, I can reach out to them.

In this case, it would be preferable to have a failing minimal testcase that reproduces this error, because the problem space is currently too large.

Given the information I have, the only reason I can think of as to why it doesn't work is because for some reason, main.js is not being copied correctly into the app. This can be checked by looking in the TestApp.app/Contents/Resources/app folder.

Just checked in ../Contents/Resources/app and I saw the main.js file was copied successfully into the app folder.  I'm going to touch base with the user to see if they have any additional steps to provide that's causing the error.

The latest update I was provided was that there were permission issues on the application contents.  The application permissions had to be manually reset and repackaged.

What sort of permission issues? What were the permissions on main.js in the package? What is the permissions on main.js in the source repo?

For both main.js in the package and source repo, the permissions were read only for everyone.  Only my development machine has read and write access to the main.js file.
The user reporting the permission issues handles the rolling out of our applications to our user base.  It appears the issue occurred when he tried to take the package and roll it into the most recent changes.  That's when he had to manually reset all of the application permissions and repackage.  I was told everything worked after that.  It may have been local to his machine when rolling in the new changes of the application where the permission issues occurred.
We are going to test more to make sure the error message won't pop up now that it's been confirmed it's working.

This doesn't sound like a bug in Electron Packager to me. I was looking at the source code of the copy method in fs-extra the other day, and it copies the permissions of the original file in addition to its contents.


Add South African Languages

This adds Afrikaans, isiZulu, and isiXhosa translations for 'five'. If anyone else speaks some of the more exotic south african languages such as Northern/Southern Sesotho, Tshivenda, Sepedi, Xitsonga, Setswana, etc. then please feel free to to submit a PR to my branch.
Also aliased isiZulu and isiXhosa to their colloquial names Xhosa and Zulu.

This pull requests needs a test.

@goosehub do you mean a test run or a unit test?

@Restioson Every pull request needs a unit test in this file. https://github.com/jackdclark/five/blob/master/test.js

I'll do that soon.

OK I actually will now...

The texts should return the output of the function being tested. For instance, for this function
five.afrikaans = function() { return 'vyf'; };
The test is currently
assert.equal('afrikaans', five.afrikaans(), 'An afrikaans five should be vyf');
But should be
assert.equal('vyf', five.afrikaans(), 'An afrikaans five should be vyf');
Also, I don't think the person running this project has accepted any PRs in a long time, so might be a dead repo. Hope this helps you in writing future tests though!

Ouch that was a typo there, did it on autopilot...


subsys: WiFi subsystem, initial structure

Contribution description
Unfortunately we cannot use RIOT as a key element for our WiFi subsystem or rather, not at this time, due to limitations of the networks where it will be.
This is a incomplete and  (very) basic structure for the WiFi subsystem, containing:
subsys-wifi /
              - main/
                    - CMakeLists.txt
                    - component.mk
                    - main.cpp
              - components/
                    - network/
                        - src/
                        - tests/
                          ...
              - CMakeLists.txt
              - Makefile
              - README.md
              - sdkconfig

Testing procedure
Using esp-idf v4.3.2  It should be built without complications as follow:
idf.py build

Issues/PRs references
Related to #96, but still a work in progress after this PR

All it's reviewed, make squash

Ready


Code quality improvements

While trying to minimize the generated C code for @NikolajBjorner:

[x] too many files include libintvector.h indiscriminately
[x] evercrypt_targetconfig.h, ibid.
[ ] kremlib (minimal) depends on compat.h which is nonsensical -- all of the stuff that mentions Prims from the machine integers should just be marked as noextract
[ ] cruft shipped in the kremlib directories: Makefile.{basic,include} and .def file
[ ] lots of stuff in types.h that should go in "full" kremlib, or maybe not even be there except for bootstrapping?
[ ] purpose of KRML_HOST_SNPRINTF?
[x] excessive inclusion of Hacl_Kremlib.h in numerous files... spurious?
[ ] too many unused variable errors -- maintaining a list in Helpers.ml that's hardcoded is not scalable, can we add an F* attribute somewhere for assumed definitions that in addition to assuming their types, assume that they are pure for the sake of triggering kremlin's unused variable elimination?

Ongoing work for unused variable elimination going on in https://github.com/hacl-star/hacl-star/pull/582 (it's tricky, so need to iterate before we have something that works just fine)


[WIP] Implement #136: Warn if current version of ROOT or NumPy is different than version built against

xref #136

Coverage increased (+0.0007%) to 99.693% when pulling 142bb4811ddc409780f97fbb4c46459e14544456 on ndawe:master into 43cc4914b5380656a5e1331fc5c9eb667b56f138 on rootpy:master.

Coverage increased (+0.0007%) to 99.693% when pulling 103fb32fbc499e6d0f9313630aacc2726c12e2ac on ndawe:master into 43cc4914b5380656a5e1331fc5c9eb667b56f138 on rootpy:master.


Scripterror

Removed ai.0.js from Script error's stack trace
Added URL custom property showing document.url to make the error more friendly

@olegsych  this commit addresses issue with Util.dump() for errors https://github.com/Microsoft/ApplicationInsights-JS/commit/30eb7c235a22527ec9ebc8fddf9386d46d7cb374

Only aesthetic comments. So, :ship: it, :sheep: it!


Range of shap_value

Hi Scott,
I am trying to explain my model using SHAP recently. The result is quite convincing, thank you for such a useful tool.
One thing that is not very clear to me is the range of 'shap_value'. Is it between -1 and 1? I took a look at Shap/explainers/kernel.py but had a hard time understand the code.

Hey! The range of the SHAP values are only bounded by the output magnitude range of the model you are explaining. The SHAP values will sum up to the current output, but when there are canceling effects between features some SHAP values may have a larger magnitude than the model output for a specific instance. If you are explaining a model that outputs a probability then the range of the values will be -1 to 1, because the range of the model output is 0 to 1. If you are explaining a model that outputs a real number or log odds the SHAP values could be larger since the model outputs can be larger. Hope that helps!

Hi Scott,
I'm also using SHAP to explain the output of my XGBoost Model and had a few questions. I was looking at your recent "Census income classification with XGBoost" ipython notebook, and I realized that shap_values are not between 0 and 1. Nor do they sum to the model predicted value. Do you why this is happening? Or, should this be happening?
The models prediction for the first value is 0.0266 and shap_value sum for this element is -3.60.

I got it, shap values are log odds!

Hello @slundberg , when you say that "the range of the values will be -1 to 1", do you mean that more features could potentially assume values of 1 or -1 as long the overall sum corresponds to the models' output?


I need to access component when click link but not change URL ?

I work on web app done by angular 7
I need when click on any item on list on have router link on nvabar.component.html   go to specified component on router link
without change URL .
navbar.component.html

Overview

Family

in app-routing.module.ts
{ path: 'overview', component: OverviewComponent },
{ path: 'family', component: FamilyComponent }

on overview.component.ts
import { CompanyService } from './../../service/company.service';
import { Component, OnInit } from '@angular/core';
import { PartDetailsService } from 'src/app/service/part-details.service';

@Component({
  selector: 'app-overview',
  templateUrl: './overview.component.html',
  styleUrls: ['./overview.component.css']
})
export class OverviewComponent implements OnInit {

  public companyProfile;
  constructor(public partDetailsService: PartDetailsService
    ,         public companyService: CompanyService) {

   }

  ngOnInit() {
console.log("welcome overview component ");
  }

so I need when click on link overview on navbar.component.html
go to component overview and show welcome on console without change URL
so
How to do that please ?
current status is when click on link overview exist on navbar it go to overview component
but it change URL
what i need is when click overview link go to component overview but not change URL ?
Example
suppose i have  URL localhost:4200/family
when click overview component go to overview component but not change URL to localhost:4200/overview meaning
link localhost:4200/family will be in URL although i click overview component and show message on console welcome overview

Hello, we reviewed this issue and determined that it doesn't fall into the bug report or feature request category. This issue tracker is not suitable for support requests, please repost your issue on StackOverflow using tag angular-cli.
If you are wondering why we don't resolve support issues via the issue tracker, please check out this explanation.


What is the real minimum deployment target?

I'm updating my app to Swift 4.2 (currently it is using Swift 4.0) but I'm getting an error in Moya-ObjectMapper that, according to the cocoapods error message, it's related to Moya.
I'm getting the error:
[!] CocoaPods could not find compatible versions for pod "Moya":
  In Podfile:
    Moya-ObjectMapper/RxSwift (= 2.7) was resolved to 2.7, which depends on
      Moya-ObjectMapper/Core (= 2.7) was resolved to 2.7, which depends on
        Moya (~> 11)

Specs satisfying the `Moya (~> 11)` dependency were found, but they required a higher minimum deployment target.

The Moya.podspec says that the minimum deployment target for iOS is 8.0. Ours is 9.0, yet I'm getting this error. The minimum deployment target is really 8.0?
Theses are the versions I'm trying to install:
pod 'Moya/RxSwift', '12.0.1'
pod 'Moya-ObjectMapper/RxSwift', '2.7'

I'm using cocoapods version 1.5.3 and Xcode 10.1
Thanks in advance :)

Hey @ivensdenner, I don't think it covers the entire issue here, but it seems like your Podfile has unmeetable versions.
Moya-ObjectMapper has Moya version defined as ~> 11 (see here), meaning it will take from Moya 11.0 up to 12, but not including 12 (so basically it will take up until Moya 11.0.2, which was our last 11.x release). Which means the highest version of Moya that can be installed is 11.0.2, but you Podfile still requires 12.0.1 😬 (more in CocoaPods docs)
So in order to use Moya-ObjectMapper and Moya 12, Moya-ObjectMapper needs to release a new version supporting Moya 12.
Does that make sense?

Oh, I see. I didn't know that ~> doesn't include the next major version. I think this is the problem :).
So I'll have to wait a new version of Moya-ObjectMapper to update Moya.
Thanks for the help @pedrovereza :)


Develop task dependency mode

for #11

Test notebook.
https://gist.github.com/taiga4112/405f91571ff13ed04887a0a668fa3613

fix #11


chore: add gho token on other test nets

Adds gho token addresses on testnet Fuji & BaseSepoia

Closing this in favor of https://github.com/bgd-labs/aave-address-book/pull/425


Update the README file

Hey @shiviboi360 I created the repo at #1 , but I could really use help with crafting a better README.


GCC 12 complained about dangling pointer

Version: 1.45.0
Platform: Unix

GCC 12.3 generated two warnings about dangling pointer:
First one:
libuv/src/unix/stream.c: In function 'uv__write_callbacks':
libuv/src/queue.h:68:19: warning: storing the address of local variable 'pq' in '((void * (**)[2])MEM[(void *[2] * *)stream_24(D) + 208B])[1]' [-Wdangling-pointer=]
   68 |     QUEUE_PREV(q) = (n);                                                      \
      |                   ^
libuv/src/queue.h:78:7: note: in expansion of macro 'QUEUE_SPLIT'
   78 |       QUEUE_SPLIT(h, q, n);                                                   \
      |       ^~~~~~~~~~~
libuv/src/unix/stream.c:941:3: note: in expansion of macro 'QUEUE_MOVE'
  941 |   QUEUE_MOVE(&stream->write_completed_queue, &pq);
      |   ^~~~~~~~~~
libuv/src/unix/stream.c:936:9: note: 'pq' declared here
  936 |   QUEUE pq;
      |         ^~
libuv/src/queue.h:40:27: note: '((void * (**)[2])stream)[26]' declared here
   40 |   ((const QUEUE *) (q) == (const QUEUE *) QUEUE_NEXT(q))
      |                           ^
libuv/src/unix/stream.c:938:7: note: in expansion of macro 'QUEUE_EMPTY'
  938 |   if (QUEUE_EMPTY(&stream->write_completed_queue))
      |       ^~~~~~~~~~~

Second one:
libuv/src/unix/linux-inotify.c: In function 'uv__inotify_fork':
libuv/src/queue.h:68:19: warning: storing the address of local variable 'queue' in '((void * (**)[2])MEM[(void *[2] * *)loop_13(D) + 16B])[1]' [-Wdangling-pointer=]
   68 |     QUEUE_PREV(q) = (n);                                                      \
      |                   ^
libuv/src/queue.h:78:7: note: in expansion of macro 'QUEUE_SPLIT'
   78 |       QUEUE_SPLIT(h, q, n);                                                   \
      |       ^~~~~~~~~~~
libuv/src/uv-common.c:507:3: note: in expansion of macro 'QUEUE_MOVE'
  507 |   QUEUE_MOVE(&loop->handle_queue, &queue);
      |   ^~~~~~~~~~
libuv/src/uv-common.c:503:9: note: 'queue' declared here
  503 |   QUEUE queue;
      |         ^~~~~
libuv/src/queue.h:40:27: note: '((void * (**)[2])loop)[2]' declared here
   40 |   ((const QUEUE *) (q) == (const QUEUE *) QUEUE_NEXT(q))
      |                           ^
libuv/src/queue.h:74:9: note: in expansion of macro 'QUEUE_EMPTY'
   74 |     if (QUEUE_EMPTY(h))                                                       \
      |         ^~~~~~~~~~~
libuv/src/uv-common.c:507:3: note: in expansion of macro 'QUEUE_MOVE'
  507 |   QUEUE_MOVE(&loop->handle_queue, &queue);
      |   ^~~~~~~~~~

This should be a false positive. We can of course diable the warning. But any other workaround?

Yes, it's a false positive. I've been seeing it, too.
I'm considering introducing a struct uv__queue type to make it go away but libuv had a type like that 10 years ago and I changed it to void *[2] to avoid leaking an internal-ish type in the public headers... oh well.
Want to send a pull request?

Yes, it's a false positive. I've been seeing it, too.
I'm considering introducing a struct uv__queue type to make it go away but libuv had a type like that 10 years ago and I changed it to void *[2] to avoid leaking an internal-ish type in the public headers... oh well.
Want to send a pull request?

Seems easy.


STORM-2250: Kafka Spout Refactoring to Increase Modularity and Testability

Making these changes based on a discussion here https://github.com/apache/storm/pull/1826 and https://github.com/apache/storm/pull/1825 about the spout getting a bit large.
This PR makes the following changes:

OffsetEntry was moved into a new class and renamed OffsetManager
OffsetManager.commit now returns numCommittedOffsets, so it doesn't have to refer to a KafkaSpout internal variable.
KafkaSpout.commitOffsetsForAckedTuples no longer iterates over acked twice. The second iteration was used to commit offsets to OffsetEntries, but we may as well use nextCommitOffsets for that iteration, since those were the offsets committed to Kafka. This also saves us a null check in OffsetManager.commit.
OffsetManager.commit no longer checks if the parameter is null. It should no longer be possible that it is null, unless there's a bug, in which case we want to throw an NPE so we can fix it.
Timer supports Storm's time simulation. This also means that it only supports time units down to milliseconds. If we need nanosecond precision, we need to update Storm's Utils.Time.
Cleaned up a few redundant version declarations in the pom. Also switched out hamcrest-all for hamcrest-core + hamcrest-library, since tests were throwing NoSuchMethodError when an assertThat failed.
Updated the tests to use time simulation, and to check if the spout calls KafkaConsumer.commitSync instead of reading the spout's internal state

I think it would be nice if RetryService also supported time simulation, let me know what you think @hmcl

I wound up keeping KafkaUnit for the tests anyway. It doesn't hurt to have some integration testing, and mocking the consumer is a bit of a hassle for no real gain.

@srdo can you please add details on any testing you've done with these changes.

@harshach Nothing beyond running the unit tests. Most of the changes are just moving code to new classes verbatim. The OffsetManager commit code changes should be getting hit by the test code.

@srdo What about calling the JIRA/PR Summary/Title: Kafka Spout Refactoring to Increase Modularity and Testability

@hmcl Renamed. I'll wait until https://github.com/apache/storm/pull/1808 has been merged to fix conflicts here, since it's likely there's some overlap

@hmcl Updated. This should be ready for review.

I'm a little unsure about whether the time unit change to Timer is a good idea. It might be better to either implement nanotime support in Storm Time, or to make the constructor parameters just be in milliseconds (I'm not sure how useful nano/microsecond support is for anyone). Anyone have any opinions on this?

@srdo everything looks OK for me. I just would rather leave the Timer in ns than in ms, and change the SimulatedTime class to support nanos. I think that moving forward that will be useful for other developers as well.

@hmcl I gave updating Time to support nanoseconds a shot. Not really sure I'm happy with it, since it requires that we also switch the current millisecond support over to using System.nanoTime (System.nanoTime and System.currentTimeMillis are not on the same timeline, and are incomparable).
If we're going to keep this change, Time.currentTimeMillis should be changed to a different name, so it doesn't imply that it's the same as System.currentTimeMillis.
@revans2 Do you have an opinion on nanosecond support in o.a.s.u.Time? (asking because you have a number of commits on the Time file)

@srdo Sorry I am kind of coming into this half way through.
History: System.nanoTime and System.currentTimeMillis have different uses and are not on the same time line.  nanoTime is guaranteed to be monotonically increasing so long as the box is up.  currentTimeMillis is not, because it is kept in sync with the system time.  currentTimeMillis on most OSes is a very cheap.  It is reading from shared memory, no system call needed.  It can be different if read from different threads even.  nanoTime, at least on x86 boxes end up reading a special register in the processor.  If there is a very small core count this is cheap.  However if you are on a system with many different cores or physical chips they all have to be synced up to guarantee that it is monotonically increasing so it ends up being about as expensive as a memory barrier.  Not super expensive but also not dead cheap.
As far as Time.java is concerned.  Simulated time can be whatever we want.  I don't care if we store nanos internally or millis internally.  It is all simulated anyways.  If we are not simulating I don't think we should mix the two or have one backed by the other.  They are separate for a reason and people most of the time will pick one or the other because of those differences.  If we want nano support then lest have Time support nano, but the milli APIs stay backed by System.currentTimeMillis, and the nano APIs are backed by System.nanoTime.

Let's back to actual usage of Timer.
        if (!consumerAutoCommitMode) {     // If it is auto commit, no need to commit offsets manually
            commitTimer = new Timer(500, kafkaSpoutConfig.getOffsetsCommitPeriodMs(), TimeUnit.MILLISECONDS);
        }
        refreshSubscriptionTimer = new Timer(500, kafkaSpoutConfig.getPartitionRefreshPeriodMs(), TimeUnit.MILLISECONDS);

Here we are taking period as milliseconds consistently, and dropping support nanoseconds on Timer means that up to 1 ms can be added to the period. Unless we're expecting users to set these config values to small value like 1 or so, I think it doesn't matter. Indeed we're having 500 ms as default value, which doesn't matter it will be 501 ms.

@revans2 I agree, it is best to keep nanos and millis separate. I've updated Time to pass through to System.currrentTimeMillis for millisecond methods, and System.nanoTime for nanosecond methods. When simulating everything gets converted to nanos.
@HeartSaVioR I agree, I'm not really seeing a use case for submillisecond support in storm-kafka-client, because the things we're currently doing with timers is to set retry backoffs or commit periods. I don't think anyone would want either to be less than 1 millisecond.
@hmcl Is there a reason nano/microsecond support is needed?
I've updated Time and Timer to support nanoseconds (hopefully without introducing bugs).

@srdo for Time and Timer for a pure correctness perspective the changes look good to me.  I don't see any issues.
I took a quick look at the rest of the patch and it too looks good, but I don't dig in to it so I don't feel comfortable giving a +1 at this point.  I would love to really take a look at it, but I am sadly swamped with other things right now.

@srdo I was also caught up in a couple of things. I will finish reviewing this tonight.

Thanks, both of you :)

@srdo Could you rebase this?

@HeartSaVioR Fixed conflict

+1 for all the KafkaSpout related code
+1 also for the SimulatedTime changes in the sense that the minor refactoring of this class does not change the semantics of the pre-existing code.
@srdo can you please squash all the commits and create a PR for 1.x-branch, such that we can backport these changes. Thanks.

Squashed this and pushed 1.x version at https://github.com/apache/storm/pull/1941. Thanks all for good feedback :)

+1


Update dev dependencies

nock, tmp, istanbul updated
Fixes #131

@ofrobots PTAL (gotta keep that badge green!)

lgtm.


Relax the PyPI package version requirements

Currently, the pip package bikeshed requires the following packages:
aiofiles==23.1.0
aiohttp==3.8.4
alive-progress==3.0.1
attrs==22.1.0
certifi==2022.12.7
cssselect==1.2.0
html5lib==1.1
isodate==0.6.1
json-home-client==1.1.1
lxml==4.9.2
Pillow==9.5.0
Pygments==2.15.1
requests==2.29.0
result==0.8.0
tenacity==8.2.2
widlparser==1.1.3

This approach (require exact version of every required package) seems uncommon for PyPI packages, since most packages describe their requirements as minimum version requirements. For example, pandas requires:
numpy>=1.23.2
python-dateutil>=2.8.2
pytz>=2020.1
tzdata>=2022.1

Requiring exact version of packages also causes many problems for bikeshed to coexist with other PyPI packages, since it is very possible that the required versions are not the latest ones, and other package may require a new version. For a real-world example, currently bikeshed 3.14.3 requires attrs==22.1.0, but latest version of jsonschema (4.18.4) requires attrs>=22.2.0. If both are >= this will not be a conflict.
Is it possible to change all requirements to >= or is there a specific reason for requiring exact versions in the Pipfile? Thanks a lot for answering in advance!

Those are exact versions now because the requirements file is autogenerated from my Pipfile.lock, which captures exact versions by design. This does cause conflicts if you install Bikeshed globally (which was previously the recommended method), but pipx (the recommended install method now) now makes it trivial to to install and manage things from pip in their own environments so they can't clash with each other in this precise way. Alternately, you can use whatever environment manager you're already using, such as pipenv.
I apologize that this is probably a change from how you were using Bikeshed before, but global installations really are dangerous, both between installed packages and, often, for system integrity itself (as OSes increasingly end up using some amount of Python themselves, and upgrading the packages they depend on can break things). The tools for managing local environments took a long time to finally become reasonably usable, but pipx is just so good now, I can't recommend it highly enough.
So, assuming you previously installed Bikeshed globally via pip, I recommend uninstalling it and using pipx to reinstall it - the docs should have sufficient instructions.

Thanks for the explanation. Now I understand the design and will definitely try the new method.


Add bool member functions to CryptoContextImpl to examine scheme type

isCKKSRNS(), isBFVRNS(), isBGVRNS(). The functions defined in scheme-id.h can be used in the code

Completed through different issues


[fix][test] Fix jvm oom on Unit Test broker group 1

Motivation
Fix the failed CI of broker group 1
The root cause is the JVM gets OOM during the test.

We are using a mock executor for test testMultiTopicsConsumerCloses which will introduce 100000 messages in the test. But the mocked executor will intercept all the tasks and add them to a LinkedList, during the test the Messages are not able to be gc.
Modifications
Use a normal PulsarClient for test testMultiTopicsConsumerCloses
Documentation
Check the box below or label this PR directly.
Need to update docs?

[ ] doc-required
(Your PR needs to update docs and you will update later)

[x] doc-not-needed
(Please explain why)

[ ] doc
(Your PR contains doc changes)

[ ] doc-complete
(Docs have been already added)

/pulsarbot run-failure-checks


Change ./Chat to TypeScript

I would like to ask that this not be merged in, and that you stop working on typescriptifying the chat until I am done refactoring it. Some of this will end up deleted, so it is maybe not that worthwhile to focus on it right now.

I would like to ask that this not be merged in, and that you stop working on typescriptifying the chat until I am done refactoring it. Some of this will end up deleted, so it is maybe not that worthwhile to focus on it right now.

Sure. I'll redo until you finish this part. BTW is that safe working on SignIn, SignUp and SignOut?

I am working on: Program, sidebar, chat

I am working on: Program, sidebar, chat

Thanks!


HDDS-4795. Separate source of Ozone container images to different repositories

This PR contains basic cleanup after moving out the source of apache/ozone-runner to this repository.
Basic Github action workflow + standard descriptors (PR template, notification) + README update.

Thanks for the review @adoroszlai. I applied the suggestions.
After merging it, I am planning to do the following next steps:

Create a similar initial commit for apache/ozone-docker-builder
Remove files from the apache/ozone-docker repository from the build/runner branches with a README which points to the new repositories
Create new tags to test the docker hub builds (master is already built well).

I also considered pushing docker images directly from the github actions to the github repository ghcr.io/.... It can make the builds faster as I assume the intra-azure traffic is faster.
I tested it, and it worked well: I created an optional step which was executed only for tags, and pushed the image. But it can work only with setting additional permissions which is pending on https://issues.apache.org/jira/browse/INFRA-20959


Add autocorrection for Rails/FilePath

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[x] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.
[ ] If this is a new cop, consider making a corresponding update to the Rails Style Guide.

I've just merge https://github.com/rubocop/rubocop-rails/pull/989. Can you rebase with the latest master branch and add a expect_correction test for that?

Thanks!


Support GPL-exported NVIDIA P2P symbols

The 550 and newer NVIDIA drivers export the P2P symbols using EXPORT_SYMBOL_GPL, which uses a different symbol table than non-GPL symbols. This modifies the nvidia-ko-to-module-symvers script such that it supports these GPL-exported symbols.

Ian.  I have merged these changes into our internal repo.  They should appear here soon.


Run module tests on try builders when flutter_tools changes

Description
Module integration tests should be run in pre-submit when the Flutter tool code changes.
Related Issues
https://github.com/flutter/flutter/pull/70790 failed build_ios_framework_module_test on post-submit.

Google testing failure is caused by infra. It should be fixed shortly. I will restart the test.


Remove dependency on squid deployment

This commit removes the usage of a proxy for web traffic from the
containers inside the podTemplate. The URL is specifying a deployment in
the webcache namespace in the AWS-int cluster. This is a piece of
infrastructure that we do not intend to maintain.
It's not clear what value the proxy was providing to the pipeline. The
Manifest stage completes in a couple minutes. The current platform tests
are failing for OAuth/TLS issues that are likely not related to the
proxy.
If this impacts other portions of the pipeline in clear and obvious
ways, we'll look at fixing those problems without reintroducing the
proxy.
This has been tested in multiple different ways. See the task for
further details.
https://phabricator.nami.run/T39254

Success will be if this PR passes the Manifests stage in the pipeline.

We passed the Manifest stage which was previously failing with "proxy unreachable" like errors.

lgtm!
I think we can skip the pipeline execution, we have already verified in the initial step the Kuberentes manifests validation is working as it should.
Thanks a lot!


_389-ds-base: <IP_ADDRESS> -> <IP_ADDRESS>

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109623 run on x86_64-linux 1

1 package built:

_389-ds-base

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 109623 run on x86_64-linux 1

1 package built:

_389-ds-base

I'm not planning to work on this anymore


-ForceChangePasswordNextLogin with Current Password?

Hi Guys,
Apologies if I'm doing this in the wrong format. First time here.
I'm trying to assist a client to have their AD Logins Force Change Password every 60 Days Without changing their current password.
I'm not sure what the correct way would be to change this policy? I was thinking to write a script to add the -ForceChangePasswordNextLogin to the user/s with the following command:
Set-AzureADUserPassword -ObjectId XXXXXXXXX -Password $currentpassword -ForceChangePasswordNextLogin True
But that doesn't work because "-Password" would need a parameter.
What would the best way be to achieve this?

This is something I've looked for as well. The -Password parameter is required.
In my case, I want to be able to immediately have a user's password expire. I was able to do so with the Set-MsolUserPassword command in the old module with the -ForcePasswordChangeOnly command, but the password parameter was not required.
I am unable to utilize the Msol module as my organization must use MFA.

Hi @Scourgeza
Thank you for your issue submission.  Unfortunately, this appears to be a problem with the cmdlt and a support issue.  Issues in this repo are for documentation issues.  You can try one of the following links for additional help.
Microsoft Support:  https://support.microsoft.com/
Azure AD forums: https://social.technet.microsoft.com/Forums/en-US/home?forum=windowsazureaditpro
Stackoverflow: https://stackoverflow.com/


MetaMask has a bug bounty

https://github.com/MetaMask/metamask-extension/issues/3127

They also instruct bug disclosers to send details to<EMAIL_ADDRESS>so I am going to list that as the security contact.


DuckDB implementation

Fixes #141
Q1-Q14 SQL queries work and passed manual cross-validation against Postgres on SF0.1
Still, many things are yet to be implemented:

[x] driver integration
[x] short queries
[x] update queries
[x] make Q13 more elegant: it should use arrays
[ ] optimization of Q14 (if possible -- it is currently very slow)
[x] rework .properties files to keep up with the changes of #203

The DuckDB passes validation when run on a single thread. However, it is not tested in CI as it takes an excessively long time.
It currently fails on a single thread, so the future work is to make it parallel: #209


Move to the official RVM or list it on RVM.io or Ruby-Lang.org?

I tried a PR on ruby-lang.org, but it seems that it was a mistake to create translations for others than English, as many of their language specific reviewers are not active anymore:
https://github.com/ruby/www.ruby-lang.org/pull/3250
I might do the PR again with English only.


[NSE-583] implement get_json_object

What changes were proposed in this pull request?
implement get_json_object
Signed-off-by: Yuan Zhou<EMAIL_ADDRESS>depends: https://github.com/oap-project/arrow/pull/49
How was this patch tested?
local tests

Test result:

This patch depends on https://github.com/oap-project/arrow/pull/54.


Readonly update

Adding all the files needed inorder to run the proj including views.py & templat dir.

an import command missed


fixes: #177

https://github.com/intel-iot-devkit/tinyb/issues/177

Hi,
I suggest to apply this change, as it is more flexible regarding file-paths on user systems, without breaking the existing setup.


Build error: cannot find -lstlink

[X] Programmer/board type: Stlink/v2
[X] Programmer firmware version: unknown
[X] Operating system: Linux Mint 17.3
[X] Stlink tools version and/or git commit hash: eea16d762979fcfc28937c784c480c8346247a3a
[X] Stlink commandline tool name: n/a
[X] Target chip (and optional board): e.g STM32F103

When attempting to compile from source, the build exits with error "cannot find -lstlink"
Attached is the output of running make release from the top directory after a fresh git clone
build.txt

Nice catch, I'm not 100% sure why this happens but on Debian 8 it works (or worked) fine. I will fix this in v1.3.1.

Related to issue #551 and probably fixed with PR #552.

works now, thanks.


Skins are not changed in IE browsers

I have faced a issue in IE browser, when I click on any skins from right panel but it is not effected in IE browser. But in all major browsers it is working fine.

this is effect write to  demo.js; you can open this file read !!

Can you please clarify how to fix this issue for ie 10. I have checked
demo.js but what should I do or write for ie. Please help
On Wednesday, November 4, 2015, CRPER<EMAIL_ADDRESS>wrote:

this is effect write to demo.js; you can open this file read !!
—
Reply to this email directly or view it on GitHub
https://github.com/almasaeed2010/AdminLTE/issues/776#issuecomment-153609267
.

--
Rajdeep Dey
Web/GraphicsDesigner

I have read the file all values are set there as variable. But What should I do. In all major browser skin changing effect is working except IE. When I see from live demo it is working fine but whenever I download the zip file and run it from my local system it is not working. IE console shows error in localstorage in demo.js

@crperlin can you please help me out from this problem

@subho1st   work for me !! ie10~ie 11 ; you must run into server!!!can local server , but can't file://path!!!
all component woking it!!!

@subho1st please make suer that your browser console shows no errors. For instructions on how to get the developer's tools, you may find them on MSDN.

thanks it is working fine now
On Wed, Nov 4, 2015 at 6:00 PM, CRPER<EMAIL_ADDRESS>wrote:

@subho1st https://github.com/subho1st work for me !! ie10~ie 11 ; you
must run into server!!!can local server , but can't file://path!!!
all component woking it!!!
[image: image]
https://cloud.githubusercontent.com/assets/8272877/10938120/ae556d9c-8332-11e5-8356-e06e5fab590e.png
—
Reply to this email directly or view it on GitHub
https://github.com/almasaeed2010/AdminLTE/issues/776#issuecomment-153705985
.

--
Rajdeep Dey
Web/GraphicsDesigner


Create cluster without dedicated K8s etcd host

How would you create a cluster using this without a dedicated K8s etcd host?
E.g. cluster with 2 nodes - a master and a worker.

I've looked into this because admittedly, saving at least $60/year/node sounds great for the non lucrative side project.
There are two ways you can squeeze the cluster - remove the dedicated etcd host and/or allow scheduling workload pods on master. If you do both, you could potentially only have a master node and would end up with a minikube.
Making these modifications is pretty easy with basic familiarity with Kubernetes architecture. For removing the dedicated etcd host, you would move the certificate provisioning in deploy.tf from the etcd block to master, and delete the etcd block. Then merge 00-etcd.yaml into 01-master.yaml. Make sure and update the master components and flannel to look for etcd at the new address. To enable scheduling on master, basically merge 02-worker.yaml into 01-master.yaml and make sure deploy.tf provisions master with the necessary certs.
I started my own branch for this kind of a cluster a while ago, but it's really sloppy and was branched before we made some important changes like securing etcd with tls... so I'm not going to publish it. If I or someone else were to approach this again, in context of #20, perhaps it would belong in sort of a "family" of repositories based around this project?

You can take a look at this branch for an example: https://github.com/nomkhonwaan/kubernetes-digitalocean-terraform


fix(p-select-dropdown): apply new vOnClickOutside

To Reviewers

[ ] Skipping review is not a problem (style, chore ONLY)
[ ] Not that difficult

Type of Change

[ ] New feature
[x] Bug fixes
[ ] Feature improvement

[ ] These changes require a usage change

[x] Others (performance improvement, refactoring, CI/CD, etc.)

Checklist

[ ] Updated Storybook documents
[x] Tested with console

Description

apply new vOnClickOutside
previous vClickOutside was not working

Things to Talk About

:tada: This PR is included in version 1.1.0-beta.413 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Multi-node training hangs at accelerator.prepare(model)

Recently I'm trying to launch multi-node distributed training using on two servers accelerate, but the training always hangs at accelerate.prepare(model). The detailed environment information and minimal code to reproduce the error are as follows
The config file of the main server(hosting the main process):
compute_environment: LOCAL_MACHINE
deepspeed_config: {}
distributed_type: MULTI_GPU
fp16: false
machine_rank: 0
main_process_ip: IP1
main_process_port: PORT1
main_training_function: main
num_machines: 2
num_processes: 2

The config file of the second server is all the same as the one above except that the machine_rank is 1 instead of 0.
The environmental information:
Python3.7.6
torch1.10.1+cu113
accelerate0.5.1

The minimal code to reproduce the code:
import torch
import torch.nn as nn
from accelerate import Accelerator

if __name__ == "__main__":
    accelerator = Accelerator()
    model = nn.Conv2d(10, 20, 3, 1, 1)
    model = accelerator.prepare(model)

then accelerate launch ./main.py
The above setting performs well on any of a single machine, but multi-node training somewhat fails. I'm pretty sure I'm using the right configuration file and the two servers can communicate with each other via the port PORT1.
btw, the two servers share the exact same runtime environment.
Any reply is appreciated : )

btw, the two servers share the exact same runtime environment.

This works on our end, so in this instance, the problem is probably some communication failure on your multi-node setup.

This works on our end, so in this instance, the problem is probably some communication failure on your multi-node setup.

Thanks for your reply. I also think so, but I have no idea what other settings should be enabled. I've enabled the PORT1 port in the main server so that the two servers could communicate with each other. Is there something I've missed?

Maybe @pacman100 who recently debugged multi-node training on our end has more ideas? On my end, nothing comes to mind.

Hello @xiabingquan , could you paste the output logs with NCCL_DEBUG=INFO before the launch command, e.g., NCCL_DEBUG=INFO accelerate launch script.py args?

Hello @xiabingquan , could you paste the output logs with NCCL_DEBUG=INFO before the launch command, e.g., NCCL_DEBUG=INFO accelerate launch script.py args?
See if the log has Network is unreachable before hanging.

Thanks for your reply :) Below is the output after adding NCCL_DEBUG=INFO
The output of the main server
~/miniconda3/envs/lrs/lib/python3.7/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  FutureWarning,
ubuntu:13025:13025 [0] NCCL INFO Bootstrap : Using enp5s0:<IP_ADDRESS><0>
ubuntu:13025:13025 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
ubuntu:13025:13025 [0] NCCL INFO NET/IB : No device found.
ubuntu:13025:13025 [0] NCCL INFO NET/Socket : Using [0]enp5s0:<IP_ADDRESS><0> [1]ens7:<IP_ADDRESS><0>
ubuntu:13025:13025 [0] NCCL INFO Using network Socket
NCCL version 2.10.3+cuda11.3

The output of the second server
~/miniconda3/envs/lrs/lib/python3.7/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  FutureWarning,
aa-ESC4000:23293:23293 [0] NCCL INFO Bootstrap : Using enp6s0:<IP_ADDRESS><0>
aa-ESC4000:23293:23293 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation

aa-ESC4000:23293:23293 [0] misc/ibvwrap.cc:63 NCCL WARN Failed to open libibverbs.so[.1]
aa-ESC4000:23293:23293 [0] NCCL INFO NET/Socket : Using [0]enp6s0:<IP_ADDRESS><0> [1]ens7:<IP_ADDRESS><0>
aa-ESC4000:23293:23293 [0] NCCL INFO Using network Socket

I am also having this same problem when attempting to run accelerate between two identical EC2 instances. Accelerate works perfectly fine for each one individually

@xiabingquan , could you try below command on both nodes before running the code and use <IP_ADDRESS> as IP1 as shown in the ens7 entry of main server.
export NCCL_SOCKET_IFNAME= ens7 .
I ran your code on distributed setup (2 nodes) but didn't face any issue. In my case, I had to set export NCCL_SOCKET_IFNAME= enp6s0 following this issue NCCL Network is unreachable / Connection refused when initializing DDP - distributed - PyTorch Forums

@xiabingquan , could you try below command on both nodes before running the code and use <IP_ADDRESS> as IP1 as shown in the ens7 entry of main server. export NCCL_SOCKET_IFNAME= ens7 .
I ran your code on distributed setup (2 nodes) but didn't face any issue. In my case, I had to set export NCCL_SOCKET_IFNAME= enp6s0 following this issue NCCL Network is unreachable / Connection refused when initializing DDP - distributed - PyTorch Forums

Thanks for your reply. Already tried that and nothing changed. I also tried running multi-node experiments via Pytorch-lightning instead of accelerate but pl got stuck in the same place. I highly doubt that it's because of my network settings but I couldn't check it in person because of COVID-19. I'll put it aside for while until I'm able to check the network settings. However, the two servers could be successfully pinged by each other server so maybe the network also works fine :( Driving me crazy

One thing to notice is that, in the main server, the last debug message is NCCL version 2.10.3+cuda11.3, but the same message didn't appear on the second server.

I have the same issues, when I use the same script and run like NCCL_DEBUG=INFO accelerate launch --config_file /tmp/config.yaml main.py, the error of master server is
NCCL_DEBUG=INFO accelerate launch --config_file /tmp/config.yaml main.py
f31e57d4-8849-429e-886f-a595642e27d6:12675:12675 [0] NCCL INFO Bootstrap : Using eth1:<IP_ADDRESS><0>
f31e57d4-8849-429e-886f-a595642e27d6:12675:12675 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
f31e57d4-8849-429e-886f-a595642e27d6:12675:12675 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE ; OOB eth1:<IP_ADDRESS><0>
f31e57d4-8849-429e-886f-a595642e27d6:12675:12675 [0] NCCL INFO Using network IB
NCCL version 2.10.3+cuda10.2
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 00/02 :    0   1
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 01/02 :    0   1
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Trees [0] 1/-1/-1->0->-1 [1] -1/-1/-1->0->1
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Setting affinity for GPU 0 to ff,ffff0000,00fffffc
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 00 : 1[1a000] -> 0[1a000] [receive] via NET/IB/0
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 01 : 1[1a000] -> 0[1a000] [receive] via NET/IB/0
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 00 : 0[1a000] -> 1[1a000] [send] via NET/IB/0
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Channel 01 : 0[1a000] -> 1[1a000] [send] via NET/IB/0
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Connected all rings
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO Connected all trees
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 8/8/512
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO 2 coll channels, 2 p2p channels, 1 p2p channels per peer
f31e57d4-8849-429e-886f-a595642e27d6:12675:12697 [0] NCCL INFO comm 0x7f5218000fa0 rank 0 nranks 2 cudaDev 0 busId 1a000 - Init COMPLETE
f31e57d4-8849-429e-886f-a595642e27d6:12675:12675 [0] NCCL INFO Launch mode Parallel

f31e57d4-8849-429e-886f-a595642e27d6:12675:12699 [0] transport/net_ib.cc:882 NCCL WARN NET/IB : Got completion from peer <IP_ADDRESS><50780> with error 12, opcode 0, len 0, vendor err 129
f31e57d4-8849-429e-886f-a595642e27d6:12675:12699 [0] NCCL INFO include/net.h:28 -> 2
f31e57d4-8849-429e-886f-a595642e27d6:12675:12699 [0] NCCL INFO transport/net.cc:459 -> 2
f31e57d4-8849-429e-886f-a595642e27d6:12675:12699 [0] NCCL INFO proxy.cc:351 -> 2
f31e57d4-8849-429e-886f-a595642e27d6:12675:12699 [0] NCCL INFO proxy.cc:452 -> 2 [Proxy Thread]
[E ProcessGroupNCCL.cpp:414] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  NCCL error: unhandled system error, NCCL version 2.10.3
ncclSystemError: System call (e.g. socket, malloc) or external library call failed or device error. It can be also caused by unexpected exit of a remote peer, you can check NCCL warnings for failure reason and see if there is connection closure by a peer.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 0 (pid: 12675) of binary: /data/miniconda3/envs/env-3.8.8/bin/python
Traceback (most recent call last):
  File "/data/miniconda3/envs/env-3.8.8/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 345, in wrapper
    return f(*args, **kwargs)
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/run.py", line 761, in main
    run(args)
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
    elastic_launch(
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
main.py FAILED

and the error of the second server is
NCCL_DEBUG=INFO accelerate launch --config_file /tmp/config.yaml main.py
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12545 [0] NCCL INFO Bootstrap : Using eth1:<IP_ADDRESS><0>
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12545 [0] NCCL INFO NET/Plugin : No plugin found (libnccl-net.so), using internal implementation
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12545 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/RoCE ; OOB eth1:<IP_ADDRESS><0>
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12545 [0] NCCL INFO Using network IB
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Trees [0] -1/-1/-1->1->0 [1] 0/-1/-1->1->-1
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Setting affinity for GPU 0 to ff,ffff0000,00fffffc
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Channel 00 : 0[1a000] -> 1[1a000] [receive] via NET/IB/0
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Channel 01 : 0[1a000] -> 1[1a000] [receive] via NET/IB/0
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Channel 00 : 1[1a000] -> 0[1a000] [send] via NET/IB/0
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Channel 01 : 1[1a000] -> 0[1a000] [send] via NET/IB/0
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Connected all rings
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO Connected all trees
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO threadThresholds 8/8/64 | 16/8/64 | 8/8/512
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO 2 coll channels, 2 p2p channels, 1 p2p channels per peer
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12566 [0] NCCL INFO comm 0x7fc730000fa0 rank 1 nranks 2 cudaDev 0 busId 1a000 - Init COMPLETE

4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12568 [0] transport/net_ib.cc:882 NCCL WARN NET/IB : Got completion from peer <IP_ADDRESS><56528> with error 12, opcode 0, len 0, vendor err 129
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12568 [0] NCCL INFO include/net.h:28 -> 2
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12568 [0] NCCL INFO transport/net.cc:459 -> 2
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12568 [0] NCCL INFO proxy.cc:351 -> 2
4a72aede-30e1-43ba-873a-de73fb9d7c94:12545:12568 [0] NCCL INFO proxy.cc:452 -> 2 [Proxy Thread]
[E ProcessGroupNCCL.cpp:414] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  NCCL error: unhandled system error, NCCL version 2.10.3
ncclSystemError: System call (e.g. socket, malloc) or external library call failed or device error. It can be also caused by unexpected exit of a remote peer, you can check NCCL warnings for failure reason and see if there is connection closure by a peer.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 0 (pid: 12545) of binary: /data/miniconda3/envs/env-3.8.8/bin/python
Traceback (most recent call last):
  File "/data/miniconda3/envs/env-3.8.8/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 345, in wrapper
    return f(*args, **kwargs)
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/run.py", line 761, in main
    run(args)
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
    elastic_launch(
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/data/miniconda3/envs/env-3.8.8/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
main.py FAILED

I found the way to solve this issue: https://github.com/NVIDIA/nccl/issues/426

Thanks a lot!!! I'll try it later : )

I found the way to solve this issue: NVIDIA/nccl#426 use command like NCCL_IB_GID_INDEX=3 NCCL_DEBUG=INFO accelerate launch --config_file /tmp/config.yaml main.py

Thanks a lot!!! I'll try it later : )

For me this didn't work on kubernetes cluster with the same error message.
The issue was that a shared memory was not available to the pod. Fixed it by adding a shared memory volume to the deployment yaml:
- mountPath: /dev/shm
name: gpus-shared-memory
volumes:
- name: gpus-shared-memory
emptyDir:
medium: Memory
sizeLimit: 128Mi
More information here: https://www.sobyte.net/post/2022-04/k8s-pod-shared-memory/


FPGA gzip area fix

Existing Sample Changes
Description
Change to GZIP design to guard against segfaults without adding predication to the kernel. This is done by adding padding to the allocated input buffers/USM pointers so that the overrun in the kernel is still within the allocated region. This allows us to remove the predication and reduce the number of LSUs. This change also includes a seed update.
Type of change

[X] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

[X] Command Line
[X] regtest infra (s10, s10 usm, a10)

Failures expected due to changes to fpga_extensions.hpp path.


Minor cleanup of guest mode for more generic guest username

Moves guest username to constants
Check if user object exists before accessing it.

:+1:


chore(e2e-test): bump build image to amd64-ubuntu20.04-20231117-7a169d0

/kind enhancement

Should we do it both for the master and 2.2?


bug: Horizontal Scrollbar in Styles Sample page(/guide/styles)

The information of width which was provided  not appropiate.
The examples which showed is in the width box like MAX_W_100, MIN_W_25, MIN_W_50, MIN_W_75 are seems same,therefore user cann't differentiate between them.
The examples which showed is in the width box like MAX_VW_75, MAX_VW_100, MIN_VW_25, MIN_VW_50 are seems same,therefore user cann't differentiate between them.

The width style sample is on hold because of the ViewWidth 100 (VW_100) horizontal scrollbar issue.

as discussed with @techoneel we have to think about a solution for specific styles in styles guide sample page.
the solution mentioned as follows:

we will provide images to showcase the UI changes for below styles (identified till now.).

POSITION
VIEW_WIDTH
VIEW_HEIGHT


Move Babel helpers into the sap.ui.define wrapper if possible

Babel replaces certain keywords with function calls when the semantics have changed between language versions. For example, typeof(id) gets transpiled to _typeof(id) and function _typeof(obj) { /* magic */ } (which handles typeof(Symbol()) === "symbol"s in older browsers) is injected near the top of the file.
The builder from @ui5/project expects that modules have an empty top level scope except for the sap.ui.define call. However, the babel-created _typeof function counts as a symbol in that scope; this yields a warning message:

WARN lbt:bundle:Builder **** warning: module app/Component.js requires top level scope and can only be embedded as a string (requires 'eval')

This is probably not a very serious error. "Requires 'eval'" refers to the UI5 loader having to eval a string read from the generated preload file, so there's no security issue either.
Still, would it be possible to move these Babel-generated functions into the sap.ui.define factory if all other code was wrapped into the factory anyways? Is this perhaps related to #15 and the injected imports?

Should be fixed by #15 also.

Looks good, the Babel helpers are now getting wrapped in the factory function as expected. 👍


Release

Hey Matt,
If it's not too much trouble, would you mind pushing a release? The work I did in #22 would help me out right about now :)
Thanks in advance!

@meowsus I think I did a release before now, if not then I'm really sorry, but one is happening in the next 12 or so mins (waiting for the CD pipeline)


pull request for e2e test at 12/25/2016 22:56:11

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


added new changes

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://672b56276f64092ff5b055de--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://672c6fff58a44809d945eae4--harness-developer.netlify.app


Broken page on https

https://craftpip.github.io/jquery-confirm/
mixed content...
Example of error message in chrome console:
Mixed Content: The page at 'https://craftpip.github.io/jquery-confirm/' was loaded over HTTPS, but requested an insecure script 'http://maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js'. This request has been blocked; the content must be served over HTTPS.

Thanks for noticing this, fixing this right away.


[BUG] Text size is more than model handle

This bug is reported by @shahrukhx01
When input text size is more than the text size model can process we get this -
(The size of tensor a (1453) must match the size of tensor b (512) at non-singleton dimension 1)

Currently we don't have proper solution hence as a hack we will truncate text to required size before passing to model.

@shahrukhx01 for time being I am turncating incoming text till we have proper solution. I will create separate ticket to handle it properly.