Cytnx Storage Refactor Proposal

Cytnx Storage Refactor Proposal

Introduction
Advantages
Class Diagrams
Proposed Changes

Templatization of Implementations of Storage and Tensor
Encapsulation of All Classes
Creation of Interfaces: class StorageBase and class TensorBase

Option 1 (Not Selected)
Option 2 (Selected)
Discussion on Removal of Types

Blind Storage from Shapes

Steps

Introduction
Per off-line (not on GitHub) discussions on 2024.10.03 and 2024.11.06, some chapters of this proposal are removed from the original unpublished draft. Only the chapters that got consensus remained.
This proposal outlines a plan for refactoring the Storage and Tensor classes, along with their respective implementation classes. The goal of this refactoring is to enhance the efficiency of the program and minimize human errors. Please note that this proposal is not finalized and does not constitute a definitive list of items to be implemented. We encourage discussions regarding any details of the proposed refactoring plan.
Importantly, this proposal DOES NOT alter the APIs for Python.
Advantages
Refactoring takes time. There is no need to refactor the code if there is no advantage. Here are some advantages we will have if we apply the plan mentioned in the following sections.

Reduced Code Volume: A smaller codebase will facilitate easier comprehension for newcomers.
Shorter Build Times: A reduced codebase will lead to faster build times.
Elimination of boost::intrusive_ptr Dependencies: Currently, we utilize two components from Boost: intrusive_ptr and unordered_map. By replacing boost::unordered_map with std::unordered_map, we can eliminate our dependency on Boost.
Decreased Human Error: By encapsulating the Storage class, we can lower the likelihood of contributors inadvertently accessing memory incorrectly.
Enabled compile time check: The proposed changes allow for more bugs to be identified at compile time, enabling the compiler to manage type conversions effectively.
Flexibility for Other Backends: While we currently support PyTorch as a backend, the current design binds the UniTensor class to PyTorch's API. This proposal aims to decouple them.

Class Diagrams
Below are class diagrams illustrating the current and proposed designs. Both diagrams omit specifiers such as const, virtual, etc., for clarity.
---
title: Current Design
---
classDiagram
  class Storage {
    +Init(...) void
    +append~T~(...) void
    +at~T~(...) T
    +to_(int device) void
    +to(int device) intrusive_ptr~Storage_base~
    +fill(const T &val) void
    +static from_vector(...) Storage
    +boost::intrusive_ptr~Storage_base~ _impl
  }
  class Storage_base {
    +Init(...) void
    +_Init_byptr(...) void
    +_create_new_sametype() intrusive_ptr~Storage_base~
    +clone() intrusive_ptr~Storage_base~
    +Move_memory(...) intrusive_ptr~Storage_base~
    +Move_memory_(...) void
    +to_(int device) void
    +to(int device) intrusive_ptr~Storage_base~
    +PrintElem_byShape(...) void
    +print_elems() void
    +real() intrusive_ptr~Storage_base~
    +imag() intrusive_ptr~Storage_base~
    +intrusive_ptr~Storage_base~ _impl
    +void *Mem
    +unsigned long long len
    +unsigned long long cap
    +unsigned int dtype
    +int device
  }
  class ComplexDoubleStorage
  class FloatStorage
  class intrusive_ptr_base~T~
  namespace boost {
    class intrusive_ptr~T~
  }
  class Tensor {
    +intrusive_ptr~Tensor_impl~ _impl
    +Init(...) void
  }
  class Tensor_impl {
    -Storage _storage
    Init(...) void
    +to_(int device) void
    +to(int device) intrusive_ptr~Tensor_impl~
    +at~T~(locator) T &
    +at(locator) Sproxy
  }
  class UniTensor {
    +intrusive_ptr~UniTensor_base~ _impl;
  }
  class UniTensor_base {

  }
  class BlockUniTensor {
    +vector~Tensor~ _blocks
  }
  class DenseUniTensor {
    +Tensor _block
  }

  Storage "1..*" o-- "1" Storage_base
  intrusive_ptr_base~Storage_base~ <|-- Storage_base
  intrusive_ptr_base~T~ <.. intrusive_ptr~T~
  Storage_base <|-- ComplexDoubleStorage
  Storage_base <|-- FloatStorage
  Tensor "1..*" o-- "1" Tensor_impl
  intrusive_ptr_base~T~ <|-- Tensor_impl
  Tensor_impl "1" o-- "1" Storage
  intrusive_ptr_base~Tensor_impl~ <|-- Tensor_impl
  intrusive_ptr_base~UniTensor_base~ <|-- UniTensor_base
  UniTensor "1..*" o-- "1" UniTensor_base
  UniTensor_base <|-- BlockUniTensor
  UniTensor_base <|-- DenseUniTensor
  BlockUniTensor "1" *-- "0..*" Tensor
  DenseUniTensor "1" *-- "1" Tensor

And here is the class diagram of the new design. The diagram only shows the stuff changed. The levels above from class UniTensor are not affected. Every interface only shows the methods to be implemented for bridging other backends.
---
title: New Design
---
classDiagram
  class intrusive_ptr_base~T~
  namespace boost {
    class intrusive_ptr~T~
  }

  class StorageBase {
    <<Interface>>
    +data() void*
    +size() size_t
    +capacity() size_t
    +resize(size_t size) void
    +reserve(size_t size) void
    +dtype() int
    +device() int
    +clone() unique_ptr~StorageBase~
    +to(int device) unique_ptr~StorageBase~
  }
  class Storage {
    +append(cytnx_variant value) void
    +at(...) cytnx_reference_variant
    +clone() unique_ptr~StorageBase~
    +to(int device) unique_ptr~StorageBase~
    +fill(cytnx_variant value) void
    -unique_ptr~StorageBase~ implementation_
  }
  class StorageImplementation~T, Allocator~ {
    +clone() unique_ptr~StorageBase~
    +to(int device) unique_ptr~StorageBase~
    +friend ostream& operator<<() void
  }
  class StorageFactory {
    <<Interface>>
    +MakeStorage(int dtype, int device): unique_ptr<StorageBase>
  }
  class StorageFactoryImplementation {
    +MakeStorage(int dtype, int device): unique_ptr<StorageBase>
  }
  class TensorBase {
    <<Interface>>
    +at(locator) cytnx_reference_variant
    +reshape_(shape) TensorBase&
    +contiguous_() TensorBase&
    +permute_(vector~size_t~ dimensions) TensorBase&
    +shape() const vector~size_t~&
    +dtype() int
    +device() int
    +clone() unique_ptr~TensorBase~
    +to(int device) unique_ptr~TensorBase~
    -data() void*
    -HasContiguousIterator() bool
    %% do not force to have an implementation of Storage
  }
  class Tensor {
    +real() Tensor
    +imag() Tensor
    +clone() unique_ptr~TensorBase~
    +to(int device) unique_ptr~TensorBase~
    -unique_ptr~TensorBase~ implementation_
  }
  class TensorImplementation~T, Allocator~ {
    +clone() unique_ptr~TensorBase~
    +to(int device) unique_ptr~TensorBase~
    +at(locator) T
    -StorageImplementation~T, Allocator~ storage_
  }
  class TensorFactory {
    <<Interface>>
    +MakeTensor(int dtype, int device): unique_ptr<TensorBase>
  }
  class TensorFactoryImplementation {
    +MakeTensor(int dtype, int device): unique_ptr<TensorBase>
  }
  class UniTensor {
    +intrusive_ptr~UniTensor_base~ _impl;
  }
  class UniTensor_base {
  }
  class BlockUniTensor {
    +vector~Tensor~ _blocks
  }
  class DenseUniTensor {
    +Tensor _block
  }

  intrusive_ptr_base <.. intrusive_ptr
  StorageBase <|.. Storage
  StorageBase <|.. StorageImplementation
  Storage "1" *-- "1" StorageImplementation
  StorageFactory <|.. StorageFactoryImplementation
  StorageFactory <-- Storage
  TensorBase <|.. Tensor
  TensorBase <|.. TensorImplementation
  Tensor "1" *-- "1" TensorImplementation
  TensorImplementation "1" *-- "1" StorageImplementation
  TensorFactory <|.. TensorFactoryImplementation
  TensorFactory <-- Tensor
  intrusive_ptr_base~UniTensor_base~ <|-- UniTensor_base
  UniTensor "1..*" o-- "1" UniTensor_base
  UniTensor_base <|-- BlockUniTensor
  UniTensor_base <|-- DenseUniTensor
  BlockUniTensor "1" *-- "0..*" Tensor
  DenseUniTensor "1" *-- "1" Tensor

Proposed Changes
Templatization of Implementations of Storage and Tensor
The current design repeatedly defines implementations for Storage, resulting in numerous duplicated functions. Additionally, Tensor_impl lacks type information, preventing compile-time type checking and leading to runtime checks that can degrade performance.
class ComplexDoubleStorage : public Storage_base {}
class FloatStorage : public Storage_base {}
...
// Tensor_impl doesn't provide the information about the type of data it contains to the compiler.
class Tensor_impl {
private:
  Storage _storage;
}

The new design templatizes the implementations of Storage and Tensor to enable type checking at the compile time. The compiler can also help to determine if one type can convert to the other type. The new design also makes the containers for CPU and the containers for GPU different types. By separating them, we can easily use the STL container, like std::vector to maintain the memory. And we only open the door of using thrust::vector to maintain the memory both on CPU and GPU.
template <typename T, tpyename Allocator>
class StorageImplementation {
private:
  std::vector<T, Allocator> storage_;
}

template <typename T, tpyename Allocator>
class TensorImplementation {
private:
  StorageImplementation<T, Allocator> storage_;
}

StorageImplementation<cytnx_int64, std::allocator<cytnx_int64>>; // Allocating the memory on CPU
StorageImplementation<cytnx_int64, GpuAllocator<cytnx_int64>>; // Allocating the memory on GPU

Encapsulation of All Classes
Using class Storage_base as an example:
class Storage_base {
public:
  void *Mem;
  unsigned long long len;
  unsigned long long cap;
  unsigned int dtype;
  int device;
  intrusive_ptr<Storage_base> _impl;
}

Makes member variables public increasing the risk of illegally accessing the memory.
void foo(Storage_base& storage) {
  // A contributor forgets to update the number of values contained.
  void* old_ptr = storage.Mem;
  storage.Mem = malloc(2, /* size corresponding to dtype */ 4);
  free(old_ptr);
}

// The other contributor uses `foo()`
Storage_base storage; // contains 10 values
foo(storage);
std::cout << reinterpret_cast<double*>(storage.Mem)[9]; // illegal access

Besides, both Google Style Guide and C++ Core Guidelines prefer private member variables.
Here is a change for class Storage_base.
template <typename T, typename Allocator>
class StorageImplementation : public StorageBase {
public:
  StorageImplementation(int dtype, int device):dtype_(dtype), device_(device) {}

  int dtype() { return dtype_; }
  int device() { return device; }
  size_t size() { return storage_.size(); } // replace `len`
  T* data() { return storage_.data(); } // replace `Mem`
  size_t capacity() { return storage_.capacity(); } // replace `cap`
  void resize(size_t count) { storage_.reize(); } // Use encapsulated interface to modify the values of `len`, `cap`, and `Mem`.
private:
  int dtype_;
  int device_;
  std::vector<T, Allocator> storage_;
}

The access control of the member variables changed or removed in the class diagram of the new design will be updated.
Creation of Interfaces: class StorageBase and class TensorBase
The introduction of these interfaces allows for a cleaner separation between interface definitions and their implementations, enhancing flexibility for future integrations with other libraries. There are two options for implementation.
Added on 2024.11.08: This proposal requests the developers to implement the type-less StorageBase and TensorBase while integrating with other libraries. However, by following with #496, it may be also possible to request the develops to implement typed StorageBase<typename> and TensorBase<typename>.
Option 1 (Not Selected)
This option proposed making existing classes base classes for implementations from other libraries but presented several issues including forward declaration requirements and mixed responsibilities.
class StorageFactory; // Need forward declaration here.

class Storage {
public:
  Storage(int dtype, int device, StorageFactory factory = StorageFactoryImplementation()) {
    implementation_ = factory.MakeStorage(dtype, device);
  }

  virtual ~Storage() = default;

  virtual cytnx_reference_variant at(size_t position) {}

protected:
  Storage() = default; // leave `implementation_` null

private:
  std::unique_ptr<Storage> implementation_;
};

class StorageFactory {
public:
  virtual std::unique_ptr<Storage> MakeStorage(int dtype, int device) = 0;
}

class CustomUint64Storage : public Storage {
public:
  // Can omit ":Storage()" because Storage::Storage() is the default constructor.
  CustomUint64Storage():Storage() {}

  // Is overriding `at()` neccessary? It is not clear because we cannot mark any
  // member function of `class Storage` as a pure virtual function.
private:
  StorageFromOtherLibrary storage_;
}

class StorageFactoryForOtherLibrary : public StorageFactory {
public:
  virtual std::unique_ptr<Storage> MakeStorage(int dtype, int device) { return new CustomUint64Storage(); }
}

// usage:
Storage storage(Type.Uint64, Device.cpu, StorageFactoryForOtherLibrary());
// storage.implementation_ is an instance of `class CustomUint64Storage`

There are three problems with this option.

There must be a forward declaration, which hides a dependency and violates the style guide.
Mixed the interface with its representation. The user who wants to use the backend provided by the other libraries will not be clear about which functions are necessary to be overridden.
Every implementation class, like class CustomUint64Storage, contains the unused variable implementation_ which wastes memory.

Option 2 (Selected)
The selected option introduces abstract interfaces (class StorageBase and class TensorBase) that must be implemented by any concrete storage or tensor class:
class StorageBase {
  ...
}
class Storage: public StorageBase {
public:
  Storage(int dtype, int device, StorageFactory factory = StorageFactoryImplementation()) {
    implementation_ = factory.MakeStorage(dtype, device);
  }

  ~Storage() = default;

  cytnx_reference_variant at(size_t position) override {}

protected:
  // With option 2, the default constructor is not needed.
  // Storage() = default;

private:
  std::unique_ptr<Storage> implementation_;
};

class StorageFactory {
public:
  virtual std::unique_ptr<Storage> MakeStorage(int dtype, int device) = 0;
}

class CustomUint64Storage : public StorageBase {
public:
  CustomUint64Storage() {}

private:
  StorageFromOtherLibrary storage_;
}

class StorageFactoryForOtherLibrary : public StorageFactory {
public:
  virtual std::unique_ptr<Storage> MakeStorage(int dtype, int device) { return new CustomUint64Storage(); }
}

// usage:
void foo(const StorageBase& storage) {
  // do something with the storage
}

Storage storage(Type.Uint64, Device.cpu, StorageFactoryForOtherLibrary());
// storage.implementation_ is an instance of `class CustomUint64Storage`
foo(storage);
foo(CustomUint64Storage()); // Encourage to use the class to tell which type of data is contained.

Except for the public member functions this proposal appeals to remove, the public member functions of class StorageBase and class TensorBase will copy from the current class Storage and class Tensor respectively. Below are the pure virtual functions of class StorageBase and class TensorBase.These pure virtual functions must be overridden by the implementation.
class StorageBase {
public:
  virtual void* data() = 0;
  virtual size_t size() = 0;
  virtual size_t capacity() = 0;
  virtual void resize(size_t size) = 0;
  virtual void reserve(size_t size) = 0;
  virtual int dtype() = 0;
  virtual int device() = 0;
  virtual unique_ptr<StorageBase> clone() = 0;
  virtual unique_ptr<StorageBase> to(int device) = 0;
}

class TensorBase {
public:
  virtual cytnx_reference_variant at(locator) = 0;
  virtual TensorBase& reshape_(shape) = 0;
  virtual TensorBase& contiguous_() = 0;
  virtual TensorBase& permute_(vector<size_t> dimensions) = 0;
  virtual const vector<size_t>& shape() = 0;
  virtual int dtype() = 0;
  virtual int device() = 0;
  virtual unique_ptr<TensorBase> clone() = 0;
  virtual unique_ptr<TensorBase> to(int device) = 0;

private:
  virtual void* data() = 0;
  virtual bool HasContiguousIterator() = 0;
}

There is no function for getting Storage from TensorBase because we don't expect other libraries to have the component equivalent to Storage. With this assumption, it means we have to change the arguments of the functions consuming class Storage to the iterators and the raw pointers.
The reason for adding HasContiguousIterator() is to support the tensor implementation which stores the data in a multi-dimensional data structure. We may consider providing alternative and less efficient algorithms, like Exp(), for that kind of tensor implementation. If we only want to support the tensor implementations storing data in a one-dimensional array, we should consider replacing the whole Storage component with vector<T> because the functionality of Storage is almost equal to the functionality of vector.
Discussion on Removal of Types
To facilitate type elimination in the C++ API, it is crucial to retain the clone() functions, as well as the class StorageBase, class TensorBase, class Storage, and class Tensor. If we only remove types during the Python binding process, these components will no longer be necessary, resulting in a cleaner and more straightforward codebase.
The usage of clone() will be further discussed in the next section.
Blind Storage from Shapes
Another area for improvement is how Storage interacts with shapes. The Storage class currently relies on shape information that is passed to its member functions by the caller, which can lead to implicit coupling between storage management and shape handling.
To foster better separation of concerns, we propose moving the functions below into Tensor from Storage.
void GetElem_byShape_v2(
    boost::intrusive_ptr<Storage_base> &out,
    const std::vector<cytnx_uint64> &shape,
    const std::vector<std::vector<cytnx_uint64>> &locators,
    const cytnx_uint64 &Nunit);
void GetElem_byShape(
    boost::intrusive_ptr<Storage_base> &out,
    const std::vector<cytnx_uint64> &shape,
    const std::vector<cytnx_uint64> &mapper,
    const std::vector<cytnx_uint64> &len,
    const std::vector<std::vector<cytnx_uint64>> &locators);
void SetElem_byShape(
    boost::intrusive_ptr<Storage_base> &in,
    const std::vector<cytnx_uint64> &shape,
    const std::vector<cytnx_uint64> &mapper,
    const std::vector<cytnx_uint64> &len,
    const std::vector<std::vector<cytnx_uint64>> &locators,
    const bool &is_scalar);
void SetElem_byShape_v2(
    boost::intrusive_ptr<Storage_base> &in,
    const std::vector<cytnx_uint64> &shape,
    const std::vector<std::vector<cytnx_uint64>> &locators,
    const cytnx_uint64 &Nunit,
    const bool &is_scalar);
virtual boost::intrusive_ptr<Storage_base> Move_memory(
    const std::vector<cytnx_uint64> &old_shape,
    const std::vector<cytnx_uint64> &mapper,
    const std::vector<cytnx_uint64> &invmapper);
virtual void Move_memory_(
    const std::vector<cytnx_uint64> &old_shape,
    const std::vector<cytnx_uint64> &mapper,
    const std::vector<cytnx_uint64> &invmapper);
virtual void PrintElem_byShape(
    std::ostream &os,
    const std::vector<cytnx_uint64> &shape,
    const std::vector<cytnx_uint64> &mapper = {});

With the new design, we can consider eliminating the entire Storage component, as it essentially functions as a one-dimensional array. This functionality can be effectively replaced by using std::vector or thrust::vector as a member variable within the Tensor component. By making this change, we can streamline the implementation and reduce complexity in the codebase.
Steps
Following these steps will enable the compiler to detect more mistakes made during the refactoring process.

[ ] Make all member variables private.
[ ] Convert class Storage_base into an abstract class.
[ ] Replace USIInit, Storage::Init(), Storage_base::Init(), and Storage_base::_Init_byptr() with constructors and factory classes.
[ ] Templatize utils_internal::Movemem_cpu_x, utils_internal::Fill_gpu, and their CUDA counterparts.
[ ] Templatize class Storage_base.
[ ] Add template information to class Tensor_Impl.
[ ] Design iterators for Storage.
[ ] Create class StorageBase and class TensorBase.
[ ] Update functions that consume class Storage or class Tensor to instead consume class StorageBase, class TensorBase, or raw pointers.
[ ] Create a custom tensor class for bridging with PyTorch.

Looks good to me -- some minor points:
StorageImplementation doesn't need to store the dtype, since that is just the index of T in the enumeration, and it certainly shouldn't be stored as an int that could get accidentally changed.  In my sketch in #496 I have a constexpr templated variable, so StorageImplementation<T, Allocator>::dtype() could be implemented simply as { return dtype_index<T>; }.
Where the data is stored (CPU, GPU etc) is effectively part of the type of StorageImplementation via the Allocator parameter, but only indirectly. It might be better to make this more explicit and extensible, with the idea that other storage devices might be added in the future, eg it might be 'stored' on another node of an MPI calculation, or on disk.
Storage_base should definitely be a template, even if nothing else is.  Replace the void* with a T*, or better still, a templated type.  Something to consider: is it possible to add a storage type that isn't directly accessible via a pointer?  The Thrust library uses device_ptr<T>, which is what ought to be used to denote the underlying storage.
I don't understand the Storage class in Option 2, it contains a std::unique_ptr<Storage>, i.e. a pointer to its own type?  How does that work?  Is the intent that Storage contains a std::unique_ptr<StorageBase> ?  In that case, Storage itself is effectively a wrapper around the pointer to StorageBase, and shouldn't inherit from StorageBase itself.
Similarly, Storage_base contains intrusive_ptr<Storage_base> _impl;, which I think cannot be intended.

Reply to @ianmccul 's comment

I will include the implementation in #496
Because this proposal use std::vector to store data, the type of allocator must be decided at the compilation time. Currently, the memory accessed by GPU is allocated with Unified Memory Access(UMA) feature. CUDA handle the transfer between CPU and GPU. So that we only have to assigned an allocator designed for CUDA at the compilation time to support both CPU and GPU computation. If we want to decide the type of allocator at runtime, the implementation will be much complex. It can be a feature in the future.
and 5. Storage_base is the current implementation. It will be dropped.
You are correct. There is a typo.

I was having another look at this.  I don't understand why classes Storage and Storage_base exist in the new design.  TensorImplementation<T, Allocator> contains a StorageImplementation<T>, so Storage and StorageBase appear to be otherwise unused, so the only reason for having them is if it simplifies StorageImplementation<T> by having the base classes.
But that isn't the case -- the code on the dev-master branch for StorageImplementation<T> is much more complicated, and many more lines of code, because it has to manage converting types to/from void* for Storage_base.  And slower too, because although StorageImplementation<T> obviously knows the type T, operations on it go via the set of overloaded virtual functions of Storage_base.  The implementations of these virtual functions are all identical -- forward to a template class anyway.
For example, from StorageImplementation.cpp,
template <typename DType>
  void StorageImplementation<DType>::fill(const cytnx_complex128 &val) {
    Fill(val);
  }
  template <typename DType>
  void StorageImplementation<DType>::fill(const cytnx_complex64 &val) {
    Fill(val);
  }
  template <typename DType>
  void StorageImplementation<DType>::fill(const cytnx_double &val) {
    Fill(val);
  }

(and so on)
There is no reason for this to be a virtual function.  Any caller of StorageImplementation<T>::fill() can just call the template function Fill() instead.  There are many other similar examples.  It seems to me that StorageImplementation<T> could be a fairly simple class that manages a T* for memory (or maybe a std::vector<T>, and TensorImplementation<T> holds a std::shared_ptr<StorageImplementation<T>> (so no need for StorageImplementation<T> to do its own reference counting).  Then Storage and Storage_base go away completely, saving a few thousand lines of code.
If you still need an untyped Storage class in the new design, then it could be done in a few ways - a good way would be as a std::variant of shared_ptr<T> objects (i.e. what the TensorImplementation<T> is holding).

Related to the above, but a separate issue,
  template <typename DType>
  template <typename OtherDType>
  void StorageImplementation<DType>::Fill(const OtherDType &value) {
    if constexpr (!std::is_constructible_v<DType, OtherDType>) {
      cytnx_error_msg(true, "[ERROR] cannot fill %s value into %s container",
                      Type.getname(Type.cy_typeid(OtherDType())).c_str(),
                      Type.getname(Type.cy_typeid(DType())).c_str());
      return;
    } else {
      if (this->device == Device.cpu) {
        utils_internal::FillCpu(this->Mem, static_cast<DType>(value), this->len);
      } else {
#ifdef UNI_GPU
        checkCudaErrors(cudaSetDevice(this->device));
        utils_internal::FillGpu(this->Mem, static_cast<DType>(value), this->len);
#else
        cytnx_error_msg(true, "[ERROR][fill] fatal internal, %s",
                        "storage is on gpu without CUDA support\n");
#endif
      }
    }
  }

I can't see any reason to defer the error here to runtime.  If you call Fill<T,U> where T and U are incompatible types, then it should be a compiler error.  The place where it becomes a runtime error is in the untyped interface, i.e. in Tensor::Fill(Scalar s).  Only at this point, do we need to determine the dtype of the Tensor and the Scalar, and then it is a runtime error if they do not match.  For example, (untested code!)
template <typename T>
class StorageImplementation {
   template <typename U> void Fill(U value); // existing Fill() function
};

template <typename T>
class TensorImplementation {
   StorageImplementation<T> storage_;
public:
   template <typename U>
   void Fill(U x) { storage_.Fill(x); }
};

class Scalar {
   std::variant<int, float, double /* other types */> data_;
};

class Tensor {
   std::variant<
       std::shared_ptr<TensorImplementation<int>>,
       std::shared_ptr<TensorImplementation<float>>,
       std::shared_ptr<TensorImplementation<double>>
       /* other types */
   > data_;

public:
   void Fill(Scalar s);
};

// Helper for Tensor::Fill (simplified version)
template <typename T, typename U>
void FillImpl(const std::shared_ptr<TensorImplementation<T>>& tensor_impl, U scalar_val) {
   if constexpr (!std::is_constructible_v<T, U>) {
       cytnx_error_msg(true, "Cannot fill type %s into tensor of type %s.",
                       cy_type_str<U>, cy_type_str<T>); // cy_type_str does not yet exist
   } else {
       tensor_impl->Fill(scalar_val);
   }
}

// Tensor::Fill implementation, foward to FillImpl via visitor pattern
void Tensor::Fill(Scalar s) {
   std::visit(
       [](auto&& tensor_impl, auto&& scalar_val) {
           FillImpl(tensor_impl, scalar_val);
       },
       this->data_, s.data_);
}

I can't see any reason to defer the error here to runtime. If you call Fill<T,U> where T and U are incompatible types, then it should be a compiler error.

The reason is that we still hold untyped Storage now. So it should be a runtime error if DType cannot constructed from OtherDType.

OK, let me think about that.  I agree at some point it needs to be a runtime error, but that should be in a function/method that belongs to the untyped interface, not a member of the typed interface StorageImplementation<Type>.  I'll look at the code and see if I can find a solution.

Hi @IvanaGyro , I had a look to see if I could find a nice way to implement Fill() (and related functions), but it ended up being a bit too intrusive for a single PR.  So I am in favour of using what you have now, as something that is a useful improvement.
However, there is a problem when compiling the code without UNI_GPU support:
        if constexpr (std::is_same_v<GpuAllocator<T>, Allocator>) {
          checkCudaErrors(cudaSetDevice(device));
          if constexpr (std::is_same_v<cytnx_complex128, T>) {
            utils_internal::cuComplexmem_gpu_cdtd(out->data(), storage_.data(), storage_.size(),
                                                  /* get_real */ false);
          } else {
            utils_internal::cuComplexmem_gpu_cftf(out->data(), storage_.data(), storage_.size(),
                                                  /* get_real */ false);
          }

Unfortunately, using if constexpr doesn't work this way.  It still needs to be syntactically correct C++ code, meaning that non-template function in both branches need to have been previously declared.  For a template function you have more freedom because you can refer to templates that would otherwise be a compile-time error to instantiate, but the compiler still needs to be able to recognise it as a syntactically valid template.
I think there is a design question here, in the sense that it is the storage that is in charge of managing these operations, but the way you have it now, it is the Allocator that determines whether you want the CPU or the GPU version.  So the storage needs to know the details of the allocator, which is not a great design.  But because it is a template, it all comes down to spelling: in effect, you have two different classes, one for CPU storage, and one for GPU storage, implemented by alternative partial specializations of StorageImplementation<T, Allocator>.  You could even make that explicit if you want:
template <typename T>
using CpuStorage = StorageImplementation<T, std::allocator<T>>;

template <typename T>
using GpuStorage = StorageImplementation<T, GpuAllocator<T>>;

So the solution to get the template to compile with and without UNI_GPU is to split the implementation into those two cases.  You could even put GpuStorage into its own header, which is conditionally included only if UNI_GPU is defined, and that way you don't have to litter the code with #ifdef UNI_GPU everywhere.
They could both inherit from a common base class, so functions that are the same in both the GPU and CPU cases can be implemented just once.  That intermediate base class could be a template of T, or the Allocator, or both.

Suppose we only support the GPU having UMA feature, is there any negative impact if we only use, no matter whether device is set to Device.cpu or Device.cuda, GpuAllocator<T> when UNI_GPU is defined? Both Storage::to() and Storage::to_() are preserved per offline discussion on 2024.10.13 (which is mentioned at the very beginning of this proposal). The most efficient way to implement "in-place" Storage::to_() is to allocate the memory only on the unified memory. If there are negative impacts to do this, we have to trade-off between Storage::to_() and the negative impacts.

Suppose we only support the GPU having UMA feature, is there any negative impact if we only use, no matter whether device is set to Device.cpu or Device.cuda, GpuAllocator<T> when UNI_GPU is defined? Both Storage::to() and Storage::to_() are preserved per offline discussion on 2024.10.13 (which is mentioned at the very beginning of this proposal). The most efficient way to implement "in-place" Storage::to_() is to allocate the memory only on the unified memory. If there are negative impacts to do this, we have to trade-off between Storage::to_() and the negative impacts.

That could work.  In this model, it isn't really a GpuAllocator, since it doesn't necessary use the gpu for computation, but just a UnifiedAllocator or similar name/concept.  This leads to the question of whether you want to implement the std::vector-based allocator vs the [Gpu|Unified]Allocator as separate classes at all, or just a compile-time switch based on UNI_GPU.  Having them as separate classes means it is more extensible, but in practice cytnx is unlikely to ever support non-unified GPU memory or anything that isn't directly readable via a pointer --being able to access *Mem is too baked in.  So in this case you could try something like
#ifdef UNI_GPU
template <typename T>
using StorageAllocator = GpuAllocator<T>;
#else
using StorageAllocator = std::allocator<T>;
#endif

Alternatively, have two versions of StorageImplementation<T>, one for the CPU and one for the GPU.  In that case, I suggest something like
#ifdef UNI_GPU
#include "StorageImplementation_gpu.hpp"
#else
#include "StorageImplementation_cpu.hpp"
#endif

as a cleaner approach than sprinkling #ifdef UNI_GPU throughout the code.  Sprinkling preprocessor tests throughout the code makes it harder to maintain, eg if you are fixing a bug in the cpu-specific part, you have to be careful to not affect the code that is conditional on the gpu, and vice versa.  But my preference would be to name them differently, so that it is always clear exactly what code is running.  Eg, if you get a bug report from someone and it has a stack trace that leads to some function in GpuAllocator then you know immediately where it was, rather than having to ask them whether UNI_GPU was configured when they installed the library (which they might not know!).
In any of these approaches, you could use an intermediate base class for common code between the cpu and gpu cases.

By the way, if you use std::vector as the backing store, you will have a the vector<bool> problem. You could try a metafunction hack to use std::vector<char> when T is bool.

I prefer one StorageImplementation<T> and only make the allocator different when UNI_GPU is defined. Using vector<char> is possible. These two changes are introduced with #469. However, #469 has some problems so I will never merge it. We have to redo some work.


Addition of a requirements file

When a user clones the repository and attempts to run the examples, packages are missing; e.g. seaborn.
Can you please add a requirements.txt from which a virtual environment can be created? Without it, installation for prospective users will be an unpleasant experience.

Thanks for the suggestion! Actually, when I look at it, seaborn, matplotlib, and pandas are just libraries I've used for plotting stuff while I hack on curry, but the only true requirements are numpy and pyquil. I've added two requirements files reflecting this in a recent commit. I'll write up some basic usage instructions in the README too, since I know the project is missing those and is probably hard to understand without them.

Sounds good. Waiting for the update so I can give more feedback.
I felt just having the requirement file is enough since I can create a virtual environment and install everything into it using pip.
P.S.: I did send you the email, using nbashige at gmail. Did you get the same?

It's been added! I got your email, but haven't had the time for a thorough reply. Expect to hear from me soon! I'll have much more time once school is over (Dec 6th)


Solution for dragEnter and dragLeave being fired on child elements

This is a solution for Issue 6, that works by keeping track of the number of times dragEnter and dragLeave were fired, and only dispatches an event when the counter reaches zero (implementing https://stackoverflow.com/a/21002544).

Accidentally wrote and ( && ) instead of or ( || ), closing this pull request.


Feature Gates: Remove CronJobTimeZone.

This PR removes the CronJobTimeZone feature gate as it becomes stable and is included in Kubernetes v1.29.
For Kubernetes <v1.29, you will need to re-enable it using the respective values.

Note: This PR is going to be tested from the CAPA v29.0.0 PR: https://github.com/giantswarm/releases/pull/1318
I therefore put the test version of the chart into cluster-aws and put the test version of cluster-aws in the release as the normal cluster test suite cannot rely on a yet unreleased CAPA version.


Add get started instructions to README.md

Codecov Report

Merging #15 (43b3190) into main (412e146) will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##                main       #15   +/-   ##
===========================================
  Coverage     100.00%   100.00%
  Complexity        41        41
===========================================
  Files             22        22
  Lines            121       121
===========================================
  Hits             121       121

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 412e146...43b3190. Read the comment docs.


Update deps

Updating zendframework/zend-code (3.3.1 => 3.3.2)
Updating j0k3r/graby-site-config (1.0.90 => 1.0.91)
Updating nikic/php-parser (v4.2.3 => v4.2.4)
Updating composer/ca-bundle (1.2.3 => 1.2.4)

Coverage remained the same at 97.662% when pulling fe6cab9a2be4032410fb8143b991659f6c2cd823 on update-deps into fd93183ef55834d963913d1f2da13732cc3c97cf on master.


Add Interface to extend TsFileResource

Description
Currently, TsFileResource serializes the PROGRESS INDEX information to TsFileResource files. When serializes the PROGRESS INDEX information, firstly write a byte 1 or 0 to represent whether contains the progress index, then write the content.
Now, we can extend the usage of the PROGRESS INDEX to store more information. We can introduce the  TsFileResourceBlockType to represent the block type of TsFileResource File and use more types bytes 2,3... to record the extended file block.
Design
https://apache-iotdb.feishu.cn/docx/VIpmddFQFoa3yKxX9MhcvZ6cnXg

Codecov Report

Merging #11362 (0c612c2) into master (3602124) will decrease coverage by 0.02%.
The diff coverage is 94.44%.

@@             Coverage Diff              @@
##             master   #11362      +/-   ##
============================================
- Coverage     48.64%   48.62%   -0.02%
+ Complexity    24413    24403      -10
============================================
  Files          2811     2812       +1
  Lines        172592   172576      -16
  Branches      20612    20604       -8
============================================
- Hits          83949    83916      -33
- Misses        88643    88660      +17

Files
Coverage Δ

...torageengine/dataregion/tsfile/TsFileResource.java
69.91% <100.00%> (+4.23%)
:arrow_up:

...ine/dataregion/tsfile/TsFileResourceBlockType.java
91.66% <91.66%> (ø)

... and 29 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Update prettierx and eslint-plugin-prettierx to utily-20231004

Change
Update prettierx and eslint-plugin-prettierx to utily-20231004
Rationale
This update is to fix the vulnerability found by dependabot (#9)
Impact
Fixes a security vulnerability
Risk

[x] The change does not increase the risk to the system and does therefore not require any extra risk analysis.
[ ] A separate risk analysis has been performed and is linked below.

Rollback

[x] Rollback is performed by reverting the merge and redeploy.
[ ] Rollback of this change requires special actions as outlined below.

I would add this to "Impact: Fixes a security vulnerability in devDependencies (offline)." to make it clear that is not a vulnerability on the production code.


How do you change the color of a button?

Is it possible to set the color scheme of for a specific button? I would like some of my buttons to have a different style so they stand out more.

no, you can't, all use the skin manager to change the color


add userInteractionEnabled for topBar && show message even StatusBar is hidden

add userInteractionEnabled for topBar ; User can set the view return by showWithStatus: userInteractionEnabled=YES to enable it.
add showEvenStatusBarHidden for JDStatusBarStyle to show message even StatusBar is hidden, the default value is NO.

Can you split this in two PRs?


how to compiling usd-unity-sdk for mac?

Does this can be compiling in mac book ? Does anyone succee in mac?

Yep -- note that OSX is supported in the package (so no need to build unless you are making modifications to C++ aspects of the library) and if you really need to build from scratch, there is an XCode project, UsdCs.xcodeproj.
Let us know if you have any issues


"Update Phone Number" when signin

Thank you, @matin for your work on the this lib. I used your code as a basis for refactoring the garmin-connect which is a typescript lib for garmin connect https://github.com/Pythe1337N/garmin-connect/pull/56
When the account has a phone number, the library works fine. However, if the account doesn't have a phone number, the POST request during the sign-in step returns an HTML page with the title "Update Phone Number", which causes the login to fail. If everything goes well, the title should be "Success".
Does anyone know how to add a phone number to a Garmin account? I tried adding a phone number on the MFA page, but I still can't see the phone number on the https://www.garmin.com/en-US/account/profile/ page.
https://www.garmin.com/en-US/account/profile/
Have phone numbers account:

Doesn't have phone numbers account:

In https://github.com/Pythe1337N/garmin-connect/pull/56/commits/e82aa730357a08b23640991843fcbe7e825eb16f

I see you're based in China, but it looks like you're using the Garmin US website. Have you tried using garmin.com.cn to enter the account?
Garmin is fairly specific about requiring requests from China to go through .cn domains. This was a fairly annoying part of the development, but a large number of the people using Garth are based in China.
I have some other ideas on what you could do, but I'd start there if, in fact, you're attempting access the account from China.

On a separate note, it's amazing to see people like you migrating the Garth logic to other languages!

I see you're based in China, but it looks like you're using the Garmin US website. Have you tried using garmin.com.cn to enter the account?
Garmin is fairly specific about requiring requests from China to go through .cn domains. This was a fairly annoying part of the development, but a large number of the people using Garth are based in China.
I have some other ideas on what you could do, but I'd start there if, in fact, you're attempting access the account from China.

Yes, I'm in China. There are many users in China who want to use Strava, but Strava cannot connect to Garmin.cn. So we tried to register an account on Garmin.com to connect to Strava, and then transfer the .fit files from Garmin.cn to Garmin.com. This way, we can use Strava.
Today, I tried to reproduce this issue by registering a new account on connect.garmin.com. When I tried to log in using Garth's method, I also got the "Update Phone Number" prompt. I then tried logging in to the account using the Connect App, and after entering my password and clicking "Log In", the "Update Phone Number" page appeared. I entered my phone number and verified it, and then logged in using Garth's method, and everything worked fine.
I hope this helps anyone who encounters the same issue.


Bar Series doesn't respect props passed as part of Data

From documentation it looks like BarSerie takes stroke (optional), fill (optional) props but it looks like it is not being passed down to actual SVG.
This can be reproduced here : https://codesandbox.io/s/6jy4py4w6w
Please let me know if I have misunderstood the doc.

I totally agree the docs might be out of sorts. You can pass style props by giving BarSeries a style prop  like <HorizontalBarSeries style={{fill: 'none}} />. For more info check the code itself: https://github.com/uber/react-vis/blob/master/src/plot/series/bar-series.js
Also, if you see problems with the docs, I would be delighted to accept a PR addressing your concerns!

@mcnuttandrew  Thanks for response.
From what I see, it makes sense to be able able to accept it from each data point, I thought it would give more flexibility to the how bar is being rendered.
For example if HorizontalBarSeries graph is being plotted using x0 and if it will everyone  flexibility to change that bar style ?


ut1999: add x86_64-darwin, aarch64-linux, i686-linux support, innoextract: enable on darwin

Description of changes

Added support for x86_64-darwin for ut1999
Added support for i686-linux for ut1999
Added support for aarch64-linux for ut1999

Did not test audio as my setup had some issues with PulseAudio dumping core. I can see from the logs ALAudio: Failed to Initialize Open AL device (alcOpenDevice).

Added support for darwin for innoextract (needed by ut1999).

For aarch64-linux it was a bit of a challenge as I run into some .so files from its System directory that were not being loaded dynamically. This led me to raise https://github.com/OldUnreal/UnrealTournamentPatches/issues/1557, but I had to close it since I cannot confirm if it's a NixOS-specific issue or something wrong upstream.
Here's a photo of it running on a PinePhone, on NixOS Mobile:

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin
[x] i686-linux (on a rebase on nixos-23.11 due to too many builds to make otherwise)

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

@Atemu , @doronbehar , please re-review.

Diff looks good, should be good to go when CI is green.


Configure rustfmt to use field initialize shorthand.

Doesn't actually change anything now, since the code already uses field initialize shorthand, but this ensures future code will continue using it.

Codecov Report

Merging #170 (d97ba7c) into dev (5c083d6) will not change coverage.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##              dev     #170   +/-   ##
=======================================
  Coverage   94.83%   94.83%
=======================================
  Files          75       75
  Lines       11292    11292
=======================================
  Hits        10709    10709
  Misses        583      583

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Crash if we try to retrieve tip for a locale that has not yet been defined

Trying to retrieve a tip for the "ja" locale crashes.  Need to use Lexik's fallback feature I think.

So we can just ask the user to define a fallback locale in the config file right? I assumed it will use the default locale as fallback.

Ideally we would define a reasonable default in our bundle and provide the user with instructions on how to override.  If it's not possible to define a default, then instructions on how to configure Lexik for this should be fine.

Let's go ahead and just add documentation to the README.  We need to do this anyway I think.  Users of our bundle will need to configure at least the following in their config:
lexik_translation:
    fallback_locale: [ <locale> ]
    managed_locales: [ <locale1>, <local2>, ... ]


Tests are failing when clean start

When checking out the project and running
docker-compose up

following error occures
aclify    | ><EMAIL_ADDRESS>test /opt/app
aclify    | > jest --coverage
aclify    |
aclify    |   console.warn node_modules/bluebird/js/release/debuggability.js:868
aclify    |     Unhandled rejection SequelizeConnectionError: Client does not support authentication protocol requested by server; consider upgrading MySQL client
aclify    |         at Utils.Promise.tap.then.catch.err (/opt/app/node_modules/sequelize/lib/dialects/mysql/connection-manager.js:149:19)
aclify    |         at tryCatcher (/opt/app/node_modules/bluebird/js/release/util.js:16:23)
aclify    |         at Promise._settlePromiseFromHandler (/opt/app/node_modules/bluebird/js/release/promise.js:512:31)
aclify    |         at Promise._settlePromise (/opt/app/node_modules/bluebird/js/release/promise.js:569:18)
aclify    |         at Promise._settlePromise0 (/opt/app/node_modules/bluebird/js/release/promise.js:614:10)
aclify    |         at Promise._settlePromises (/opt/app/node_modules/bluebird/js/release/promise.js:689:18)
aclify    |         at Async.Object.<anonymous>.Async._drainQueue (/opt/app/node_modules/bluebird/js/release/async.js:133:16)
aclify    |         at Async.Object.<anonymous>.Async._drainQueues (/opt/app/node_modules/bluebird/js/release/async.js:143:10)
aclify    |         at Immediate.Async.drainQueues (/opt/app/node_modules/bluebird/js/release/async.js:17:14)
aclify    |         at runCallback (timers.js:672:20)
aclify    |         at tryOnImmediate (timers.js:645:5)
aclify    |         at processImmediate [as _immediateCallback] (timers.js:617:5)

this is because the versions are not fixed in the docker-compose and sequalize seems to not be capable to connect to mysql:8 for now
i will send a pull request to fix this issue in a minute

It will be merged into the next release (v1.6.0).
Thank you @matthiaseigner !


Update package test to use new strong mode compatible expectAsyncN me…

Does anyone have tips on best practices to debug failing flutter tests. Looks like some changes in package:test between0.12.15+4 and 0.12.18+1 break Flutter test.

When I dug into this before, it looked like some (all?) of those issues had been fixed in the master branch of package:test.  We should validate that and then ask the maintainers to push a new version.

I'm assuming the current PR is not for review and is just to have Travis test it. Please let us know when it's ready for review. Thanks!

@jacob314 I'm going to close this PR just to get it off our list of PRs waiting for review, but please do not hesitate to reopen it or file a new one when you're ready for review. Thanks!


Consumer in the output view rather than editor

Hi
This is just random idea but what is the reason behind using editor to consume messages? Syntax highlighting?

Closing as duplicate of #53

Closing as duplicate of #53


Snackbars don't go above the bottom bar with attachShy()

I give my snackbar the root CoordinatorLayout and it was showing above the bottom bar when using just attach(this, ...) (as described in the issue  "Snackbars should go above the bottom bar. #36" ) but when I now use attachShy() it overlaps the bottom bar.

I'm in same situation.
I had problem using BottomBar.attach(this,...) using CoordinatorLayout (the view doesn't match the parent) so I user BottomBar.attachShy(...) and view was ok, but snackbar overlaps bottom bar yet.

@roughike I got a strage case related this problem.
I'm studying your library and I see this tutorial: with 3 elements all is ok bottom bar is grey and snackbar doesn't overlap bar, but when I add 2 elements I get a lot of space above toolbar (like this issue), bottom bar becomes blue and snackbar overlaps it.

@roughike This is due to CoordinatorLayout adding the snackbar at z order that makes your BottomBar have lower z index than the snackbar view. I had this issue in my lib. Take a look how I solved it here.  PreLollipopBottomNavWithSnackBarImpl inner class. Basically, calling child.bringToFront() allowed me to ask CoordinatorLayout to grant my layout same z order as before just at same moment the snackbar is added to the CL.

@roughike I didnt have issues on APIs that support elevation because the bottom bar should be always above snackbar as said in specs. On older apis I had to reorder the layout to front so Snackbar now is below my layout.

@chemickypes Fixed in PR #111. @roughike please review it

@roughike This is basically, what I said in the comment. The code is copied from my library and works flawlessly with yours. :)


Networking documentation missing

Describe the bug
The "Networking with host" document is correctly rendered in 1.14
https://docs.zephyrproject.org/1.14.0/guides/networking/networking_with_host.html
but in latest it is not
https://docs.zephyrproject.org/guides/networking/networking_with_host.html
It will print
Sorry, Page Not Found

Never mind, I did not notice that the "latest" link is not correct. BTW, the page https://docs.zephyrproject.org/1.14.0/guides/networking/networking_with_host.html gives incorrect link to latest document for some reason.


[Foundation] 1-ary IndexPath forms invalid range on slices

This is a cherry pick of pr #10311
1-ary IndexPaths incorrectly implemented slicing for ranges of 1..<1
This resolves https://bugs.swift.org/browse/SR-5098.
Explanation:
IndexPath had a missing case for 1..<1 in 1-ary variant storage when slicing ranges. This commit adds that missing case (since integers are not naturally exhaustive and fall into a default case they were hitting the "out of range" cases)
Scope:
This is only runtime behavior for IndexPath and is a un-common use case (but still something that we should definitely not let fail)
Radar (and possibly SR Issue):
https://bugs.swift.org/browse/SR-5098.
rdar://problem/32619380
Risk:
This is relatively low risk since it is only a runtime behavior and only changes an un-common case.
Testing:
A unit test was added testing both the reported case of dropFirst() as well as the root cause of slicing.

@swift-ci please test


Immutable JSON?

Original comment, but see next comment: The experimental implementations of JSONPatch operations on the JSON class are an interesting and useful idea, but they prevent users of JSON instances from having any reliable expectation of immutability.
Would it make sense to split these out into a MutableJSON mixin?

On further thought, immutability isn't guaranteed even without the operations (obviously... was not thinking it through yesterday), so maybe an immutable mixin is more of a possible future enhancement

immutability isn't guaranteed

Indeed, to be strictly immutable it would really have to define a __new__ constructor rather than __init__.
I did spend a lot of time staring at the JSON class and thinking 'this could / should be immutable' and wondering how to enforce immutability even for JSONSchema (inlining some of the bootstrap keywords, for example, so that JSONSchema.uri etc wouldn't have to be set from the outside). Having an immutable base class and complex, mutable derived classes gets complicated. Interestingly, it was my jschon-translation package that finally put the nail in the coffin of immutability - instances of class JSONTranslationSchema(JSONSchema) hold state that is dynamically updated by custom keywords. This in turn encouraged me to go forth and implement __setitem__, __delitem__ and insert on JSON, which I had always envisaged as useful features for a JSON class.

Makes sense — thanks for the explanation!


Popup window to make a new post

Can we get a popup window to make a new post in the app?
When scrolling through hundreds of new posts, our only option now is that link which scrolls to the top of all posts to make a new post. Instead can we have a button called "New post" which does a popup window to make a new post, so we don't lose our place in the stream trying to play catch up?
Thanks for all your hard work!

Already released. Closing this


Disable when hardhat-coverage is in use

The hardhat-coverage plugin modifies contract code, and also the associated ABIs.  ABI export should be disabled when this plugin is in use.
See also: https://github.com/ItsNickBarry/hardhat-contract-sizer/issues/8

released in 2.8.0


can i have ur contact?

i want u help me to improve this project

Yes sure
On Tue, Sep 7, 2021 at 10:45 AM muslichwr @.***> wrote:

i want u help me to improve this project
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kiraAdi/ATTENDANCE-MANAGEMENT-SYSTEM/issues/1, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ATPIIIZBWPFIFHONWDROBFDUAWNXPANCNFSM5DRSF75Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

will u help me ? ur project can't run in my software


Add demo papers to demo_papers.csv

Number of Volunteers: 1
Follow-up on #8, we will need to add demo papers to the papers.csv file.

[x] Get information from demo chairs (after #74 is done)

@hao-fang Hi, I can do this. Is there an google sheet about demo papers available yet?

@hao-fang are demo papers considered main conference papers? So do we just add them to papers.csv or we do a separate page?

Closing this with #171
I opened a follow-up issue #172.


Fix bash script name

Bash script name should use underscores, not hyphens

[ ] Update bash script help section with the new bash script name
[ ] Update README to use the new bash script name


Allow parens outside an IIFE - aka dogballs ftw

One of Crockford's style points that the community convention mostly disagrees with is IIFE style:
(function(){ ... ])()  // community
(function(){ ... ]())  // crockford

I'm surprised the jshint community hasn't proposed this yet, but I'd like to see an option in jshint to allow that paren orientation.
Been thinking about this for a while, but haven't posted the ticket as naming this particular option..  But in honor of how Crockford talks about it in this youtube video...
I propose we name this option dogballs

Why not just allow it in the same way the other is allowed. I am for just allowing with no option name.

That also works for me.

Related to https://github.com/jshint/jshint-next/issues/2.

Just allowing both variations isn't such a good solution.
We should force to write consistent code - no matter which format. So I think we need an option for it.

+1 with option, dogballs being default. Consistency is king!

@antonkovalyov for sure - I don't want to add such an option into this (freezed) version ;)
But the style enforcer should force one of the many possibilities (I count 4 so far (which are in use), +function() {}()). So it's more or less an extension to option immed.

it seems the immed option controls this?  if you set immed: true it enforces crockford's style.

@neonstalwart You are correct.

@antonkovalyov i'm not sure why you reopened this.  with immed: false you can have dogballs or crockford's.  do you think there needs to be something more than that?  maybe @paulirish just didn't realize this option exists.

@neonstalwart immed: true should support both variations since its description is:
This option prohibits the use of immediate function invocations without wrapping them in parentheses. Wrapping parentheses assists readers of your code in understanding that the expression is the result of a function, and not the function itself.
And I believe both dogballs and hidden dogballs should be treated equally under this option (or we will have to move the option to the list of deprecated crocklamations).

oh ok fair enough - i was just looking at it from the perspective of "how do i make this thing stop complaining?"


Process keeps running in background

The image-roll process keeps running in the background after I close the window. It only happens if I use the open menu. It doesn't keep running in the background if I use the command-line to open an image or if I double-click on an image.
If I run the version compiled from source then the process is at least alive after 10 minutes (I manually ended the process after that).
The flatpak version does end the process eventually after 40 seconds. I suspect bwrap or something else external ends the process.
Operating system: Fedora 35
Version: Flatpak 1.3.1 and debug main 5dc0253e145257026b9fb3cd89b1d9bc8ebe028a
Steps to reproduce:

Open Image Roll from the menu or from the command-line (do not double-click an image).
Click the hamburger menu button and click "Open...".
Select an image and click "Select".
Click the X to close the window.
Open a terminal and run ps -aux | grep image-roll
Notice that the image-roll process is still running.

Again, the flatpak version seems to eventually stop after 40 seconds but a version compile from source doesn't seem to ever stop.
Thanks for writing a Rust image app by the way! I'm excited to be able to run more Rust  code for daily tasks :smile:

If I move the file_chooser.destroy() call out of signal handler then the bug goes away. I don't understand why. If the destroy call is not queued then it might be destroying something that is still being used, which would result in undefined behavior. Either way, it is fine to destroy it after the file_chooser.run() call because it "blocks" (it enters a recursive main loop more specifically [1]). That would be the simplest solution:
file_chooser.connect_response(move |file_chooser, response| {
    if response == gtk::ResponseType::Accept {
        let file = if let Some(file) = file_chooser.file() {
            file
        } else {
            post_event(&sender, Event::DisplayError(anyhow!("Couldn't load file")));
            return;
        };
        post_event(&sender, Event::OpenFile(file));
    }
});

file_chooser.run();
file_chooser.destroy();

Another option is to not use a signal handler. The code already "blocks" on the file_chooser.run() call:
if file_chooser.run() == gtk::ResponseType::Accept {
    let file = if let Some(file) = file_chooser.file() {
        file
    } else {
        post_event(&sender, Event::DisplayError(anyhow!("Couldn't load file")));
        return;
    };
    post_event(&sender, Event::OpenFile(file));
}

file_chooser.destroy();

[1] https://gtk-rs.org/gtk3-rs/stable/latest/docs/gtk/prelude/trait.NativeDialogExt.html#tymethod.run

I think that the file chooser is not fully destroyed when it runs in the signal handler, most likely that "zombie" file chooser keeps the app open after closing the window.

I have merged your commit, I have also found similar problem in save as file chooser dialog which will be fixed in another commit.

Yeah that sounds plausible. Thanks!


sync event isn't triggered when a success callback is provided

Around line #327
      var success = options.success;
      options.success = function(resp, status, xhr) {
        //...
        if (success) {
          success(model, resp);
        } else {
          model.trigger('sync', model, resp, options);
        }
      };

I'd like to bind to sync as well as accept a callback. If this behavior is intentional I'm just curious why.
PS. Thanks for the amazing framework, we love & use a variety of documentcloud projects at Art.sy.

I'm curious about this as well.  I can't seem to find a discussion on why 'sync' is not fired when a success callback is provided...
Also, any reason not to implement 'sync' with the promise returned by $.ajax?

The behavior is intentional -- the idea being that you use a specific success callback to override the default evented behavior ... you can always trigger the event yourself from within the callback if you like, but the reverse is not true.
That said, we can certainly change it if y'all think there are more compelling use cases for the other way around.
Finally, after Art.sy launches to the general public, I hope you'll submit it for the Backbone homepage.

I think that always triggering the event would be less confusing - isn't the event orthogonal to the success callback? Perhaps a "silent" property on the options argument could be used to suppress the event if required?


Added prop to disable dragging on the Y axis

dragY prop can be used to disable dragging on the Y axis to restrict cards to movement on the X axis. Defaults to true so that current behaviour is unchanged.

Thanks @nick-michael !


Allow keras.Variable for loss weights

Fix #20294
With this PR, we can use keras.Variable for loss weights:
import numpy as np

import keras
from keras import layers
from keras import models

x = np.random.rand(8, 3)
y1 = np.random.rand(8, 1)
y2 = np.random.randint(0, 2, (8, 1))

inputs = layers.Input(shape=(3,), name="input_a")
output_a = layers.Dense(1, name="output_a")(inputs)
output_b = layers.Dense(1, name="output_b", activation="sigmoid")(inputs)
model = models.Model(inputs, [output_a, output_b])

loss_weights = [
    keras.Variable(initializer="ones", shape=()),
    keras.Variable(initializer="ones", shape=()),
]
model.compile(
    optimizer="sgd",
    loss=["mean_squared_error", "binary_crossentropy"],
    loss_weights=loss_weights,
)

loss_weights[0].assign(0.00001)
loss_weights[1].assign(1.0)
model.fit(x, (y1, y2), batch_size=2, epochs=1)

# Change the values
loss_weights[0].assign(1.0)
loss_weights[1].assign(0.00001)
model.fit(x, (y1, y2), batch_size=2, epochs=1)

4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - loss: 0.7180 - output_a_loss: 6.9707e-06 - output_b_loss: 0.7180
4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - loss: 0.6000 - output_a_loss: 0.5999 - output_b_loss: 6.9904e-06

We can see that the individual losses correspond to the loss weights.

Codecov Report
Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 57.44%. Comparing base (e80fe5b) to head (b600c6d).

Files with missing lines
Patch %
Lines

keras/src/trainers/compile_utils.py
33.33%
0 Missing and 2 partials :warning:

:exclamation:  There is a different number of reports uploaded between BASE (e80fe5b) and HEAD (b600c6d). Click for more details.
HEAD has 6 uploads less than BASE

Flag
BASE (e80fe5b)
HEAD (b600c6d)

keras
4
1

keras-torch
1
0

keras-tensorflow
1
0

keras-jax
1
0

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #20306       +/-   ##
===========================================
- Coverage   78.83%   57.44%   -21.40%
===========================================
  Files         511      511
  Lines       48989    48990        +1
  Branches     9022     9022
===========================================
- Hits        38621    28141    -10480
- Misses       8505    19037    +10532
+ Partials     1863     1812       -51

Flag
Coverage Δ

keras
57.44% <33.33%> (-21.26%)
:arrow_down:

keras-jax
?

keras-numpy
57.44% <33.33%> (+0.01%)
:arrow_up:

keras-tensorflow
?

keras-torch
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Thanks for the PR! Can you add as a unit test something similar to the code snippet you posted? Once thing I'd like to guard for, is to make sure that the arrays attached to those variables can pass the compilation boundary and can be modified afterwards.

Thanks for pointing that out. It seems the following snippet works fine with tf and torch but fails with jax:
import numpy as np

import keras
from keras import callbacks
from keras import layers
from keras import models

x = np.random.rand(8, 3)
y1 = np.random.rand(8, 1)
y2 = np.random.randint(0, 2, (8, 1))

inputs = layers.Input(shape=(3,), name="input_a")
output_a = layers.Dense(1, name="output_a")(inputs)
output_b = layers.Dense(1, name="output_b", activation="sigmoid")(inputs)
model = models.Model(inputs, [output_a, output_b])

class LossWeightsAdjuster(callbacks.Callback):
    def __init__(self, loss_weights):
        self.a = loss_weights[0]
        self.b = loss_weights[1]

    def on_epoch_begin(self, epoch, logs):
        self.a.assign(float(epoch % 2))
        self.b.assign(float((epoch + 1) % 2))

loss_weights = [
    keras.Variable(initializer="ones", shape=()),
    keras.Variable(initializer="ones", shape=()),
]
loss_weights_adjuster = LossWeightsAdjuster(loss_weights)
model.compile(
    optimizer="sgd",
    loss=["mean_squared_error", "binary_crossentropy"],
    loss_weights=loss_weights,
)
hist = model.fit(
    x, (y1, y2), batch_size=2, epochs=2, callbacks=[loss_weights_adjuster]
)
output_a_loss = hist.history["output_a_loss"]
output_b_loss = hist.history["output_b_loss"]
np.testing.assert_allclose(output_a_loss[0], 0.0)
np.testing.assert_allclose(output_b_loss[1], 0.0)

The root cause might be that the states in callbacks doesn't propagate to *_step in JAXTrainer.
Should we support this feature? I may need to add some codes in JAXTrainer.

I think this is too difficult and messy to support in JAX.
What I would suggest for this workflow is to update the weights in Python (as floats) and recompile the model when you want the new weight values to be taken into account.

Got it. This PR should be closed now.


Clean up project template

Fix spelling / typos
Avoid hardcoding SGX_SDK, so that it can be overridden from the environment
cargo fmt

Cool, change applied: 92a4b6b..c553179

Thank you so much! Merged.


Change default support link

Feature Request 🛍

Use Case

When the add-in has some problem or if a user gets any problem with the add-in, the user will click on the get support option here
Scenario 1: When the add-in is working properly, but the user notices some error

Scenario 2: when the add-in gets the error

but currently, we have contoso.com/help as the default support link
https://github.com/accordproject/cicero-word-add-in/blob/2caf0f5abf6b05e0ff1cc6d54da355e45147540c/manifest.xml#L10
Also, https://www.contoso.com/help redirects to https://www.microsoft.com/en-us so the user might get confused about where to get the support
Possible Solution

A possible solution will be to redirect the user to this link:
https://github.com/accordproject/cicero-word-add-in/issues/new?assignees=&labels=&template=custom-issue.md&title=
So that he can create an issue on GitHub and then maintainers can help him...
I was also thinking of pointing the user to some FAQ's where he might get his queries resolved but currently, we are not having any documentation hence pointing the user to create an issue would be better
I would like to make PR for this if this is a valid issue :)

You can submit a PR for this.

As I had suggested that we may point the user to the create issue link because as of now we are not having any documentation or any FAQ's for Add-in ( I would also like to work on this afterward)
So, I was thinking of a custom-issue template and bug-report template link but then there are some heading like Possible solution / Possible implementation which normal add-in user should not bother about, also we may need to know the Word Version as this is never asked in those templates but we may need that here

So I was thinking to create a new issue template for this that would look similar to this

Bug Report 🐛

Expected Behavior

Current Behavior

Steps to Reproduce

Context (Environment)

Desktop

OS: [e.g. macOS]
Browser: [e.g. Chrome, Safari]
Browser's Version: [e.g. 0.22.15]
MS Word Version : [e.g 16.0]

If we don't think there is a need for this new issue-template then maybe we can just update bug-report.md and include MS word version in Desktop Environment
@algomaster99 can you share your thoughts on this?


Updated download_array to properly read from literal string using StringIO wrapper

This line below is causing deprecation warnings to appear wherever this line is used, namely in the cleanlab studio multiannotator tutorial.
pd_data: pd.DataFrame = pd.read_json(res_json[name], orient='records')
The following line in the multiannotator tutorial is what is returning the deprecation warning:
pred_probs = studio.download_pred_probs(cleanset_id)
The warning can be seen below:

this seems like a high-stakes change, so please review it carefully @axl1313 . Thanks!


Webhook Issue after adding the TWc api

I updated the cloud function with the new covid-webhook containing TWC APi, but when i try to check stats for country:Nigeria, it returns total number instead. It worked fine before, but I get an error this time.
AM I not setting the parameters right?

I later got the solution.
Changed country parameter to location.
Thanks


[Docs] update README and add new badges

在 README 中新增 GitHub Release 和 GitHub Downloads badges，同时新增了开放原子声明
Checklist

[ ] 在 https://yuque.com/tsdoc 增加或更新了必要的文档
[ ] 增加或更新了必要的测试用例
[ ] 对于重要修改，更新了CHANGES文件
[ ] 当前修改存在对已有API参数或返回值的改变
[ ] 当前修改存在对旧版本功能的兼容性改变（如网络协议或密码算法）

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Error in png.cpp

Error in following example
https://github.com/qPCR4vir/nana-demo/blob/master/Examples/png.cpp#L14
should be:
#elif defined(NANA_ENABLE_JPEG)

Thank!
Fixed, https://github.com/qPCR4vir/nana-demo/commits/cmake-dev and https://github.com/qPCR4vir/nana/commits/cmake-dev
but now I have tested jpeg... and have problems:  a definition duplication of boolean in window.h vs jpeg @cnjinhao ??

In file included from C:/MinGW/include/jpeglib.h:31,
                 from C:\Prog\ExtLib\nana\source\paint\detail/image_jpeg.hpp:10,
                 from C:\Prog\ExtLib\nana\source\paint\image.cpp:28:

C:/MinGW/include/jmorecfg.h:242:13:

error: conflicting declaration 'typedef int boolean'
 typedef int boolean;
             ^~~~~~~

			 In file included from c:\mingw\x86_64-w64-mingw32\include\wtypes.h:8,
                 from c:\mingw\x86_64-w64-mingw32\include\winscard.h:10,
                 from c:\mingw\x86_64-w64-mingw32\include\windows.h:97,
                 from c:\prog\extlib\nana\source\detail\mswin/platform_spec.hpp:24,
                 from c:\prog\extlib\nana\source\detail\platform_spec_selector.hpp:18,
                 from C:\Prog\ExtLib\nana\source\paint\image.cpp:16:

c:\mingw\x86_64-w64-mingw32\include\rpcndr.h:65:25:

note: previous declaration as 'typedef unsigned char boolean'
   typedef unsigned char boolean;
                         ^~~~~~~

I just checked the jconfig.h coming with the nuwen's mingw distro, it is different with libjpeg-turbo and libjpeg. The libjpeg-turbo and libjpeg define boolean if __RPCNDR_H__ is not defined, so they work well with windows.h.
The libjpeg in nuwen's mingw doesn't check if the boolean has been defined by standard headers. To solve the problem, base on comment at jmorecfg.h:235, adding -DHAVE_BOOLEAN seems to work fine.

hi!,
yes, I tested that and compile, but i got a run-time error about the user expecting 600 b but jpeg expecting 633 b ??? because now boolean is char and not int ??

The boolean in nuwen's libjpeg is int, but Windows.h, libjpeg and libjpeg-turbo provide boolean as unsigned char.
I think the best solution is to let people avoid using the nuwen's libjpeg.


Create ads and meta images with new design in mind

I think if there is one that fits all, or at least one that fits the aspect radio of these (with some bleed):

Pixels are doubled for retina screens.
According https://sproutsocial.com/insights/social-media-image-sizes-guide
cc @amritnagi

If I remember correctly, this is done?


MAINT: Package the license file

Adds the license file to MANIFEST.in so it will be included in sdists and other packages. This is done both to ensure compliance with the license terms and also for the benefit of end-users.

Sloppy, never noticed that that was missing. Thanks @jakirkham

I think it was packaged before, but it just got dropped after the license got renamed from COPYING to LICENSE.


doom don't work ,Warning: Package cl is deprecated

System information:

```
SYSTEM  type       darwin
	config     x86_64-apple-darwin19.5.0
	shell      /bin/zsh
	uname      Darwin 20.1.0 Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101 x86_64
	path       (./bin ~/bin /usr/local/sbin ~/.dotfiles/bin /opt/homebrew/bin /usr/local/bin ~/hsg/flutter/bin ~/.pyenv/bin /usr/local/heroku/bin /go/bin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Library/Apple/usr/bin ~/.emacs.d/bin /Applications/Emacs.app/Contents/MacOS/Emacs ~/hsg/hugo/dotfiles/swift-sh/bin /Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin<EMAIL_ADDRESS>EMACS   dir        ~/.emacs.d/
	version    27.1
	build      Nov 14, 2020
	buildopts  --disable-dependency-tracking --disable-silent-rules --enable-locallisppath=/usr/local/share/emacs/site-lisp<EMAIL_ADDRESS><EMAIL_ADDRESS>--with-xml2 --with-gnutls --without-dbus --with-imagemagick --with-modules --with-rsvg --with-ns --disable-ns-self-contained
	features   RSVG IMAGEMAGICK GLIB NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS JSON PDUMPER LCMS2 GMP
	traits     (batch server-running envvar-file)
DOOM    dir        ~/hsg/hugo/iDoom.d/
	version    2.0.9
	build      grafted, HEAD -> develop, origin/develop, origin/HEAD 110d70b 2020-11-19 00:50:57 -0500
	elc-files  0
	modules    (:input chinese :completion company ivy :ui doom doom-dashboard doom-quit hl-todo modeline nav-flash neotree ophints (popup +all +defaults) treemacs unicode vc-gutter vi-tilde-fringe window-select workspaces :editor (evil +everywhere) file-templates fold multiple-cursors rotate-text snippets :emacs dired electric ibuffer vc :term shell :tools docker (eval +overlay) :checkers syntax spell :tools (lookup +docsets) lsp :os macos :tools (magit +forge) tmux upload :lang (cc +lsp) data emacs-lisp markdown (org +dragndrop +hugo +ipython +pandoc +pomodoro +present) plantuml sh (&user swift +lsp) :app (rss +org) :boyer (&user my-org) :config (default +snippets +bindings +commands +smartparens))
	packages   ((lsp-sourcekit) (swift-helpful) (swift-playground-mode))
	unpin      (n/a)
	elpa       (n/a)
```

Duplicate of #3372.
Please us github search in the future.

I install emacs27 on Macmini ARM
arch -x86_64 brew tap d12frosted/emacs-plus
arch -x86_64 brew install emacs-plus

Is this warning causing the doom to fail to load?
Why is there no such information:
Doom loaded 222 packages across 50 modules in 9.230s

Try launching Emacs in debug mode: emacs --debug-init

emacs --degbug-init
Debugger entered--Lisp error: (file-missing "Cannot open load file" "No such file or directory" "all$
  require(all-the-icons)
  eval-buffer(#<buffer  *load*-553830> nil "/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." n$
  load-with-code-conversion("/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." "/Users/boyer/.e$
  require(doom-modeline-core)
  eval-buffer(#<buffer  *load*> nil "/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." nil t)  $
  load-with-code-conversion("/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." "/Users/boyer/.e$
  doom-modeline-mode()
  run-hooks(after-init-hook delayed-warnings-hook)
  command-line()
  normal-top-level()
Package cl is deprecated

This means one of the needed packages are missing. Try deleting ~/.emacs.d/.local/straight/repo/all-the-icons then run doom sync.

YES,

I delete all-the-icons.el then run doom sync

$ ls ~/.emacs.d/.local/straight/repos/all-the-icons.el
Cask                   all-the-icons-faces.el fonts
LICENSE                all-the-icons.el       logo.png
README.md              data                   test

emacs --debug-init
print the same info:

Debugger entered--Lisp error: (file-missing "Cannot open load file" "No such file or directory" "all$
  require(all-the-icons)
  eval-buffer(#<buffer  *load*-553830> nil "/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." n$
  load-with-code-conversion("/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." "/Users/boyer/.e$
  require(doom-modeline-core)
  eval-buffer(#<buffer  *load*> nil "/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." nil t)  $
  load-with-code-conversion("/Users/boyer/.emacs.d/.local/straight/build-27.1/d..." "/Users/boyer/.e$
  doom-modeline-mode()
  run-hooks(after-init-hook delayed-warnings-hook)
  command-line()
  normal-top-level()
Package cl is deprecated

I delete this cache:~/.emacs.d/.local/straight/build-27.1 then doom sync.
this ok.
thanks.

after update emacs27 ，the icon shows this way, I don't know what caused the error.


Feature: allow to generate requirements.lock for each modules in a workspace

When using a workspace with multiples modules, all packages are installed in a top-level .venv with a requirements.lock that combine all packages. It's good for local developpement
But when you want to build a Docker image for each module, you want to minimize the size of your build, so you want to only install the package necessary for each module of the workspace. Currently you can use RYE and docker very easily with:
RUN sed '/-e/d' requirements.lock > requirements.txt
RUN pip install -r requirements.txt

The issue is that, in workspace we only have a top-level requirements.lock.
Feature suggestion: when using a workspace, we should add the option to generate a per-module requirements.lock
Like:
my-workspace/
├─ module1/
│  ├─ src/
│  ├─ pyproject.toml # Module/Package pyproject.toml for build/publish
│  ├─ requirements.lock
│  ├─ requirements-dev.lock
├─ module2/
│  ├─ src/
│  ├─ pyproject.toml # Module/Package pyproject.toml for build/publish
│  ├─ requirements.lock
│  ├─ requirements-dev.lock
├─ pyproject.toml # Workspace pyproject.toml for the IDE
├─ requirements.lock
├─ requirements-dev.lock

I quite like the idea but that could be quite slow. Might be a good idea to brainstorm how this would be configured and how it would behave.

I quite like the idea but that could be quite slow.

I might be missing something, but I don't think it will be much less performant. It would just be writing a file per each of the workspace projects.

I would like to propose my idea and maybe even work on it (my rust knowledge is really basic, but I would love to give it a shot):
Configuration:
Under [tool.rye.workspace], there can be a new configuration key: per_member_lock: bool (for example) and that will toggle the functionality for whether to write the lockfile per member or not.
Behavior:
The per member locks would be purely symbolic, they wont be used to install any requirements, that would be all handled with the top level lock
Found this issue, as it happens to be the exact use case I would like to have in rye and found missing (usecase is a monorepo), and would love to aid and implement it if I can!

Rye does appear to know which sub-projects need which modules, since it documents the references in the lockfile itself:
# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: ["all"]
#   all-features: false
#   with-sources: false
#   generate-hashes: false

-e file:src/project-a
    # via project-b
-e file:src/project-b
accelerate==0.26.1
    # via project-a
    # via transformers
aiohttp==3.9.5
    # via project-b
    # via fsspec
    # via instructor
    # via litellm
...

Although it would require the sub-projects to correct declare their depencies, rather than relying upon the universe of installed packages being sufficient, this does seem like the sort of thing that could be determined by pruning the dependency tree. (Honestly, you might not need to get THAT sophisticated if you parsed the lockfile comments and build the tree from that...)

Update:
First, I see that there is a PR open to fix this, and thanks @davfsa for taking a stab at it!
I the meantime, I have managed to hack this together as follows:
Imagine I have a workspace with project_a, project_b, and project_c.
Each have their own separate dependencies from pypi. Rye does a good job of ensuring that they have versions of each dependency in common with the global lockfile.
Now imagine that project_b depends on project_a, and project_c depends on project_b
In each sub-project's pyproject.toml, I list dependencies for that project of course, as well as dependencies on other locally-installed package in my workspace. I just reference them by name though! I don't reference them by any sort of relative path. (This works fine because every time we need to resolve that package at package install time, it's already been installed for us by rye or another mechanism anyway.)
We do it this way because pyproject.toml files don't allow for relative references. requirements.txt files do though.
So this is where it starts to get a bit hacky.
As a sibling of each pyproject.py, I've got a requirements.in file, that looks like this.
# project_b/requirements_local.in

# this line is here because in my case I want my lockfile to record the fact that I'm installing my local dir in editable mode
# -e file:./

# this line is here because I need to ensure that project_a is installed when compiling the lockfile. That will automatically install all of project_a's deps referenced in pyproject.toml
-e file:../project_a

# and this says: Make sure that you also include any other local references to projects that project_a might have.
-r ../project_a/requirements_local.in

Now I can go into project_b and run:
WORKSPACE_DIR=$(git rev-parse --show-toplevel)
FOLDER=src/project_b

RELATIVE_PATH_FROM_FOLDER_TO_WORKSPACE=$(realpath --relative-to="$FOLDER" "$WORKSPACE_DIR")
rye run uv pip compile \
  --constraint "$RELATIVE_PATH_FROM_FOLDER_TO_WORKSPACE/requirements.lock" \
  --strip-extras \
  ./pyproject.toml \
  ./requirements_local.in \
  -o requirements.lock

(You can do a standard pip compile instead, but uv pip compile is about 100x faster.)
This tells pip compile:
1: Use requirements from both pyproject.toml AND requirements_local.in
2: When you are generating a lockfile for this project, use the root requirements.lock as a "constraints" file: https://pip.pypa.io/en/stable/user_guide/#constraints-files
That way, we respect the resolved versions of packages from our root lockfile.
3: Put the output into a local requirements.lock
It will also include the references to the editable installs of the local packages, which you might not want:
# requirements.lock:

# This file was autogenerated by uv via the following command:
#    uv pip compile --constraint ../../requirements.lock --strip-extras ./pyproject.toml ./requirements_local.in -o requirements.lock
-e file:../project_a
    # via -r ./requirements_local.in

If you don't want that sort of thing, you can add another option to the uv pip compile like:
--no-emit-package project-a
That way, the reference to project-a itself isn't in the lockfile, but its deps still are. This is equivalent to pip's --unsafe-package flag, which has a bad name for historical reasons.

If you want to (ab)use this lockfile a bit more and make sure that your project-specific lockfile is sufficient, you can create a sub-venv with that lockfile you made:
[tool.rye.scripts]
venv_create = "uv venv ./.venv"
venv_sync= "uv pip sync --python ./.venv/bin/python ./requirements.lock"
venv_create_sync = {chain = ["venv_create", "venv_sync"]}
check_deps = "basedpyright --project ./pyproject.toml"

[tool.basedpyright]
include = ["src/"]
# Ensure that it uses the .venv that we created for this project with the lockfile
venvPath="./"
venv=".venv"
# We really only care about some import issues, so we disable everything and report on missing imports:
typeCheckingMode = "off"
reportMissingImports = true

Now you can run in your sub-project:
cd src/projectB && rye run venv_create_sync && rye run check_deps


feat(ast): visit TSTypeQuery

type Bar = typeof Foo;

Current dependencies on/for this PR:

main

PR #2021   👈

PR #2022

This stack of pull requests is managed by Graphite.


Able to Unbond when Pending

After bonding to the community node, I expected not to be able to unbond in the same round that I bonded
The unbond button should not appear if the node is pending
A few minutes after bonding, i hovered over the community transcoder node and the unbond button appeared
Not sure if you should be able to see the unbond button, or when you click on it maybe it says error, you cannot unbond during the same round . But maybe I bonded to that node and its for a round starting tonight, so that is why it let me unbond?
🐞Steps to Reproduce

Without having ever bonded before, bond to node

🐞Your Environment

chrome
macbook

Fixed by e777536c5aee5d5781ebab4b036e01dbbc184f42
(https://github.com/livepeer/livepeerjs/commit/e777536c5aee5d5781ebab4b036e01dbbc184f42#diff-168f7607e9231d79576400718086977eR177)
This fix has been pushed live to https://explorer.livepeer.org.


search() and pulldata() support?

Just wondering if enketo-transformer supports the search() and pulldata() functions for pre-loading external data from a CSV?
These functions are described here: http://xlsform.org/#pre-loading-csv-data
I notice that after I run the example XLSForm through enketo-transformer with media: { 'fruits': './itemsets/fruits.csv' } in the options, I don't see any reference to that path, or the options in the output form / model...

Sorry, this is the XForm I'm attempting to transform:
<?xml version="1.0"?> <h:html xmlns="http://www.w3.org/2002/xforms" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:h="http://www.w3.org/1999/xhtml" xmlns:jr="http://openrosa.org/javarosa" xmlns:orx="http://openrosa.org/xforms" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> 	<h:head> 		<h:title>in</h:title> 		<model> 			<instance> 				<in id="in"> 					<fruit/> 					<flavour/> 					<note_fruit/> 					<meta> 						<instanceID/> 					</meta> 				</in> 			</instance> 			<bind nodeset="/in/fruit" type="select1"/> 			<bind nodeset="/in/flavour" readonly="true()" type="string"/> 			<bind nodeset="/in/note_fruit" readonly="true()" type="string"/> 			<bind calculate="concat('uuid:', uuid())" nodeset="/in/meta/instanceID" readonly="true()" type="string"/> 		</model> 	</h:head> 	<h:body> 		<select1 appearance="search('fruits')" ref="/in/fruit"> 			Choose a fruit 			<item> 				name 				<value>name_key</value> 			</item> 		</select1> 		<input appearance="pulldata('fruits', 'flavour', 'name_key', 'mango')" ref="/in/flavour"> 			Flavour Profile 		</input> 		<input ref="/in/note_fruit"> 			The fruit  				<output value=" /in/fruit "/> 				pulled from csv 		</input> 	</h:body> </h:html>

Hi Matt,
pulldata() is definitely supported and works for both XML and CSV data. It requires a secondary instance in the XForm definition. See https://opendatakit.github.io/xforms-spec/#secondary-instances---external. Though note that doesn't cover CSV data which goes like this:
<instance id="countries" src="jr://file-csv/country-data.csv"/>

In Pyxform, this will be added automatically when pulldata() is used anywhere in the form. Try opendatakit.org/xlsform to convert because some implementations are using a very old version of pyxform (I think KoBo hasn't updated e.g.).
search() appearance is not supported and will never be supported in Enketo. The syntax makes no sense whatsoever. It is an unfortunate rogue feature. However, you can do what it does and a lot more with pure XPath (or with select_one_form_file and select_multiple_from_file in XLSForm) .
Note that CSV support has been added to Enketo like this: https://enketo.org/xforms/#csv-support.
This still in the proposal stage in the ODK XForms Spec. There is actually no official support for CSV in that spec... Sorry, this situation is very annoying and messy. One day, we'll get this sorted!

Thanks for the detailed response!
I changed my XLSForm to the Enketo-way:

type
name
label
calculation

select_one_from_file   fruits.xml
fruit
Choose a fruit

calculate
flavour
Flavour Profile
instance('fruits')/root/item[name=${fruit}]/flavour

note
note_fruit
The fruit ${fruit} pulled from csv

I then have the following "fruits.xml" document:
<root xmlns:jr="http://openrosa.org/javarosa" xmlns:odk="http://www.opendatakit.org/xforms" xmlns:orx="http://openrosa.org/xforms"> 	<item> 		<name>mango</name> 		Mangoes 		<flavour>Sweet</flavour> 	</item> 	<item> 		<name>organge</name> 		Oranges 		<flavour>Sour</flavour> 	</item> 	<item> 		<name>strawberry</name> 		Strawberries 		<flavour>Tart</flavour> 	</item> </root>
The formatting in GitHub is rough, but the idea is each item has a <name>_</name>_<flavour>_</flavour>.
I'm passing in media: { "fruits.xml": "./fruits.xml" } to the enketo-transformer. However, in the model that gets spit out from enketo-transformer, I see:
<instance id="fruits" src="./fruits.xml"> 		<root xmlns:jr="http://openrosa.org/javarosa" xmlns:odk="http://www.opendatakit.org/xforms" xmlns:orx="http://openrosa.org/xforms"> 			<item> 				<name>_</name> 				_ 			</item> 		</root> 	</instance>
I assume that enketo-core will resolve and read in the XML file at the time the form is rendered.
For... implementation-specific reasons, this doesn't actually work for me directly. I'm planning on replacing the <instance id="fruits" src="./fruits.xml"> node in the model XML that enketo-transformer returns directly with the contents of the fruits.xml file before loading the enketo-core. That should work right?

Ah yes, the dummy content is there to fool ODK Validate. I believe that is no longer necessary an should probably be removed (in pyxform):
<root ...>
    <item>
        <name>_</name>
        _
    </item>
</root>

I assume that enketo-core will resolve and read in the XML file at the time the form is rendered.

yes, indeed

I'm planning on replacing the  node in the model XML that enketo-transformer returns directly with the contents of the fruits.xml file before loading the enketo-core. That should work right?

Yes, that should work, as long as you also remove the src attribute from the instance to prevent Enketo from trying to populate it again upon load. So it becomes an internal instance as far as Enketo's form engine is concerned. The only difference between an internal and external instance is that initial attempt to populate it. All the form logic (e.g in your calculate) is the same.

Some progress: https://forum.getodk.org/t/proposal-customize-name-and-label-columns-in-selects-from-external-data-in-xlsform/27265/2

Hi Martijn and Helene!
Thanks a lot for sharing this! Great to see this moving forward!
I have no account on that website so I will add my two cents here, and sorry if it is not
relevant or just unrealistic:
From a user's point of view, I think it would be more immediate to understand if the names
of the columns were set in the choice sheet, like the name of the CSV value column in the
choice sheet's "name" column and the name of the CSV label column in the choice sheet's
"label" column. Hopefully, this would further in time allow to set different CSV columns for
different languages, using the columns "label::en", "label::de" etc.
Ideally, this would also allow to set a filtering column in the "filter" column or basically to map
any column from the choice sheet to a column of the CSV file. (I can't speak for XML files,
which I have never used.)
Thanks for your constant support!
Cheers!
Le jeudi 16 juillet 2020, 22:33:31 EEST Martijn van de Rijdt a écrit :
Some progress: https://forum.getodk.org/t/proposal-customize-name-and-label-columns-in-selects-from-external-data-in-xlsform/27265/2[1]
—You are receiving this because you commented.Reply to this email directly, view it on
GitHub[2], or unsubscribe[3].[4]

[1] https://forum.getodk.org/t/proposal-customize-name-and-label-columns-in-selects-from-external-data-in-xlsform/27265/2
[2] https://github.com/enketo/enketo-transformer/issues/44#issuecomment-659624434
[3] https://github.com/notifications/unsubscribe-auth/
AGFGE3V7F7TIBXOJ7MJNL2TR35IYXANCNFSM4FJWLZ5A
[4] https://github.com/notifications/beacon/
AGFGE3SOBE4VEHMBXO2NUH3R35IYXA5CNFSM4FJWLZ5KYY3PNVWWK3TUL52HS4DFVREXG4
3VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOE5IRD4Q.gif


hexByObfuscatingString fails due to XOR anomaly

I added Obfuscator-iOS to my project via CocoaPods and followed the directions to obfuscate strings. I'm trying to use hexByObfuscatingString: to obfuscate a secret key for GitHub's API, but I keep getting an error that it could not obfuscate my secret key string.
I walked through the code and I think that the problem is at line number 164 in Obfuscator.m. In the reveal: method, there's this line of code:
*dataPtr = *dataPtr ^ *keyPtr;

The problem that I am seeing is that both *dataPtr and *keyPtr point to the character '8' at one index. Therefore, '8' XOR '8' is going to be byte 0. Because the code is operating on C strings, this '0' byte is prematurely terminating the obfuscated string and the obfuscated version appears to be truncated.

Hi Michael,
It's not actually a bug in the code. In fact, it's the key to how the obfuscator works!
What you are seeing is a limitation of the technique. The only way around it is to change your salt (i.e. add more classes) and try again:
See https://github.com/pjebs/Obfuscator-iOS/blob/master/README.md#L125
If you instead use the technique listed under "More Advanced Usage", you will notice that generateCodeWithSalt method will automate the process of obfuscating all your strings. This is the preferred method to use

Thanks for responding. Yes, I realized as I was looking at the code more that there is really no way around this given how the code is currently implemented because it uses reveal: for both obfuscation and reversal.
If you ever re-write that part, it might be a good idea to use different methods for obfuscating and reversing. Instead of modifying the C-string in-place, you could use an NSMutableString to build the obfuscated string and implement some kind of marker for a zero byte.

I think you are right. For the Next version I will do that


Friendly error and version

I added an error is an appname is not found and added --version.

I personally like the idea of displaying a message, and allowing the user to choose from the list of apps.

We could throw and error and list the apps that are available. My concern with continuing is that the .okta_aws_login_config never gets cleaned up.

It looks like @epierce has this covered in his PR. We should merge his and I’ll rebase mine.

This should be rebased with master and ready to go. It’s just adding —version and a few syntax cleanups