Adding Properties/Modifiers

So in the spirit of this playground, I've started trying to add it to a real project to see how far this takes me.
I have created a CircleStyleLayer based on the SymbolStyleLayer - all is working well, and thanks to the Macros, adding the properties is less work then without them. But even for my limited use case I already see that there's going to be a LOT of boilerplate code to support all properties that the underlying MLNCircleStyleLayer supports - and any addition to MLNCircleStyleLayer will need an update to CircleStyleLayer...

So my question is, is this the right approach? Wouldn't it be wiser to take this a step further and auto-generate these properties/modifiers from the definitions of MLNCircleStyleLayer etc themselves? Or is this something the current Swift cannot help us with? Maybe this is the goal already and I'm just stating the obvious, in which case, sorry - but wanted to contribute to any discussion 😊

Thanks for your involvement @hactar! It's great to have others using it!

But even for my limited use case I already see that there's going to be a LOT of boilerplate code to support all properties that the underlying MLNCircleStyleLayer supports - and any addition to MLNCircleStyleLayer will need an update to CircleStyleLayer...

Yes, by my count it's going to be dozens if not hundreds of properties per layer type to get to complete support :/

So my question is, is this the right approach? Wouldn't it be wiser to take this a step further and auto-generate these properties/modifiers from the definitions of MLNCircleStyleLayer etc themselves? Or is this something the current Swift cannot help us with? Maybe this is the goal already and I'm just stating the obvious, in which case, sorry - but wanted to contribute to any discussion 😊

Yes, a lot more auto-generation is definitely the long-term goal. Recently I haven't had as much headspace to devote to this, but here's a quick brain dump from me:

I haven't done a large-scale property/modifier bridging effort yet precisely because I am not 100% sure the current approach is correct. In particular, I'm questioning whether the current macro attachment point is optimal or whether we can make things better another way.
I actually TRIED to test a few alternate approaches (I had a peer macro in my working copy), but...
a. At the time (a few months ago), Xcode / LLDB were broken and I couldn't actually debug the macros (which meant zero development, as there are basically no docs for macros at the moment; WWDC videos heavily feature LLDB cuz that's the only way to get a sense of what you're transforming haha)
b. While MapLibre Native 6.0 is out for iOS, but it doesn't yet support macOS. This makes it incredibly difficult to build macros against due to the way that Xcode / SPM work at the moment, which led us to separate macros out into a separate project temporarily 😂 This is a drag on iterative development speed.

Jumping back to your direct question about auto-generation, at the moment, Swift macros don't give you compile-time type introspection of obj-c class hierarchies. They are just syntax transformers. So this is not presently possible. However, I have a very half-baked thought in the back of my head about writing an actual preprocessor that generates most of the Swift code in this project (I guess from the Obj-C), and uses macros for the heavy lifting, but I have not yet given too much thought to that.
Please feel free to brainstorm on this thread :) cc @Archdoog.

Alright - so for the time being, this is the right approach. I'll submit a PR for CircleStyleLayer later then, it won't cover all possible modifiers, but will include the ones I need for my project :) - leaving issue open in case Archdoog still wants to comment, but feel free to close, my question has been answered, thanks!

Great; thanks! And yes, I'd love a PR for your circle style layer. I think as more people start building more with it we'll hit a point where it makes sense to spend more time on the automatic generation (figure out what we want the APIs to look like first; then figure out how to scale it).
If anyone has any ideas, feel free to either open an issue with a proposal, or I think discussions are also enabled for more open-ended stuff :)


FAISS load_local() file path issue.

System Info
Langchain Version : [0.0.190]
Python Version : 3.10.7
OS : Windows 10
Who can help?
@eyurtsev @hwchase17
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[X] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
Error: RuntimeError: Error in __cdecl faiss::FileIOReader::FileIOReader(const char *) at D:\a\faiss-wheels\faiss-wheels\faiss\faiss\impl\io.cpp:68: Error: 'f' failed: could not open D:\Question Answer Generative AI\Langchain\index_files\ThinkPalm☺\en\index.faiss for reading: Invalid argument
Try creating a file path :  D:\Question Answer Generative AI\Langchain\index_files\ThinkPalm\1\en
mainly some folder name with just a number, then try to load that file path using faiss.load_local(). The path created after passing string path to Path(path) class is causing some issue.
Expected behavior
Issue: The actual file path is : D:\Question Answer Generative AI\Langchain\index_files\ThinkPalm\1\en
File Path created by PurePath : *D:\Question Answer Generative<EMAIL_ADDRESS>

I tried renaming the file name 1 to 2 and still had that issue, then I tried naming it to 1A and still had the issue. But when i renamed it to A1 then the loading of the index worked. So there is some sort of path name issue when the name of the folder starts with a number.

I found out that the issue is caused because the string is read as a normal string literal instead we should input the path as a raw string literal.
path = Path(r"D:\Question Answer Generative AI\Langchain\index_files\ThinkPalm\1\en")
WindowsPath('[D:/Question](file:///D:/Question) Answer Generative AI/Langchain/index_files/ThinkPalm/1/en').
But not everyone passes the path as a raw string literal so we need to fix this.

Please provide a code snippet for your solution. I am still facing the same issue.

@AlvinKimata see my last comment thats how I solved it. If you are passing the path as a string variable pass it as a raw string by adding r before double quotes.
path = Path(r"D:\Question Answer Generative AI\Langchain\index_files\ThinkPalm\1\en")

I am still not able to resolve the issue.
I noticed that using the write_index method from faiss reproduced the issue above :
from langchain.vectorstores.faiss import FAISS
import faiss

store = FAISS.from_texts(splits, embedding_function)
faiss.write_index(store.index, '/content/faiss_index')

As a workaround, I used the save_local method from the FAISS library within the langchain framework. This allowed me to save the FAISS index to a file for later use.
from langchain.vectorstores.faiss import FAISS

store = FAISS.from_texts(splits, embedding_function)
store.save_local('/content/faiss_index')

#Load file.
new_db = FAISS.load_local('/content/faiss_index', embedding_function)

Didn't know about this write_index method . I was using the save_local and load_local from the start.

you can use allow_dangerous_deserialization=True like
metallica_saved = FAISS.load_local(r"C:\Hatem\Work\faiss\faiss_index_metallica", embedding,allow_dangerous_deserialization=True)
and that sould solve the problem


Changing the allowed maximum conversation length in Plato-2

Plato-2 removes all the conversation data points which exceed its constraint of conversation length. How can I increase this allowed conversation length so as to incorporate context lengths?
Please do help, I have been trying to do this for a while now. But everytime it seems like the pretrained weights that is being downloaded for the 24L model has a fixed unique vocabulary size, which leads to the training to not happen.
Thank you.

Using tool to expand position embedding: https://github.com/PaddlePaddle/Knover/blob/develop/knover/tools/expand_embedding.sh
sh knover/tools/expand_embedding ${PATH_TO_MODEL_DIR}

And then update training config file:
Add one line after https://github.com/PaddlePaddle/Knover/blob/6040f5831535c0130f3f54fc048d9bec80259c93/projects/PLATO-2/finetune/24L_train.conf#L30 such as:
train_args="--max_src_len 384"


typst-lsp: 0.9.4 -> 0.9.5

Diff: https://github.com/nvarner/typst-lsp/compare/v0.9.4...v0.9.5
Changelog: https://github.com/nvarner/typst-lsp/releases/tag/v0.9.5
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 252370 run on x86_64-linux 1

2 packages built:

typst-lsp
vscode-extensions.nvarner.typst-lsp


Update config with 1.0 extras

Could have sworn I could create a multi-file PR through the GitHub UI, can't seem to figure it out at that moment. This just adds some configurable niceties introduced in Netlify CMS 1.0.

Thanks for doing this! 👍


Private members included in API hash confuse name hashing

The #2441 made private members of traits to contribute to the API hash but left behavior of name hashing undefined. The current implementation doesn't include private members in name hashes. For example:
trait A {
  // uncomment B
  // private object B
}

The overall api hash of A will change if you uncomment B but the hash of the name A will remain the same. This breaks a contract in incremental compiler that says there's always at least one name changed if api has changed (or its hash). Nothing blows up at the moment because that contract hasn't been enforced but it's enough to add
assert(modifiedNames.regularNames.nonEmpty || modifiedNames.implicitNames.nonEmpty,
    s"Modified names for $modified0 is empty")

to the sbt.inc.NamesChange and run trait-private-object scripted test to see:
[info] java.lang.AssertionError: assertion failed: Modified names for A is empty
[info]  at scala.Predef$.assert(Predef.scala:179)
[info]  at sbt.inc.NamesChange.<init>(Changes.scala:30)
[info]  at sbt.inc.IncrementalNameHashing.sameAPI(IncrementalNameHashing.scala:33)
[info]  at sbt.inc.IncrementalCommon.sameClass(IncrementalCommon.scala:146)
[info]  at sbt.inc.IncrementalCommon$$anonfun$6.apply(IncrementalCommon.scala:129)
[info]  at sbt.inc.IncrementalCommon$$anonfun$6.apply(IncrementalCommon.scala:129)

I've run into this problem while debugging class-based dependency branch.
The unenforced contract can be seen here:
https://github.com/sbt/sbt/blob/0.13/compile/inc/src/main/scala/sbt/inc/IncrementalNameHashing.scala#L23
  override protected def sameAPI[T](src: T, a: Source, b: Source): Option[APIChange[T]] = {
    if (SameAPI(a, b))
      None
    else {
      val aNameHashes = a._internalOnly_nameHashes
      val bNameHashes = b._internalOnly_nameHashes
      val modifiedNames = ModifiedNames.compareTwoNameHashes(aNameHashes, bNameHashes)
      val apiChange = NamesChange(src, modifiedNames)
      Some(apiChange)
    }
  }

The code assumes that if the api is not the same it's because some names has changed.

@Duhemm, I think the best way to fix this problem is to introduce a new type of API change TraitPrivateMembersModified and have a separate hash for private members to catch this is indeed the change you're looking at when invalidating dependencies.
Having a separate type of the API change makes sense because you want only to invalidate dependencies by inheritance from traits. The member ref dependencies shouldn't be invalidated. At the moment they're not because the set of changed names is empty so member reference invalidation becomes a dummy operation.

To unblock my test of class-based dependency tracking, I'll revert hashing of private members for now.

Related https://github.com/pantsbuild/pants/issues/4185.

@jvican is this in your list of SC zinc issues? I don't see it under https://github.com/sbt/zinc/issues/212, but i see this issue under the 1.0.0 milestone, so I think it should be, maybe?

@jvican Ping again. Is this on your radar? Is there a ticket (or tickets) for this sbt/zinc side? Is this fixed or worth fixing for zinc 1.0?

It is super important. It's one of the first things in my Zinc list. You can close it, there's a related ticket in sbt/zinc.

I don't want to close it until/unless we can provide a forward link in sbt/zinc. Just for paper-trail reasons.

There's such a forward link, have a look 6 comments before this one.

For clarity: https://github.com/sbt/zinc/issues/118 apparently


Add code to seqr-pipeline to inject seqr's HAIL_SEARCH_DATA_DIR into the pipline as SEQR_APP_HAIL_SEARCH_DATA_DIR

Is assigning HAIL_SEARCH_DATA_DIR env var in the lib chart instead of hail-search chart a valid solution here?


Replace deprecated exists? methods

Due to the deprecation of the exists? methods for the File and Dir classes in ruby 2.1, and eventual removal in ruby 3.2, this pull request replaces the calls to these methods with the "bare verb" exist? method.
See Ruby 3.2 removals in the changelog.

+1. Can we get this PR looked at?


A few menu display tweaks

This pull request includes:

a few adjustments to the CSS to make the look for mouse (as opposed to keyboard) users more consistent with the current MathJax menu without interfering with the keyboard usage (I hope).

The focus outline is not shown if the menu is opened by mouse (but focus is still set).
The focus outline is not shown for dialog boxes opened by mouse (but focus is still set).
The focus outline is not shown for close buttons when clicked by mouse
The focus is not shown on the math element after a menu or dialog is closed by mouse click (but is if it is closed by keyboard action).

So the only significant difference for mouse users from the current UI is that clicking on a math element focuses it and shows the focus outline.  (Note that we probably need to implement something for <maction> elements so that they can get the focus and be activated by keyboard and screen reader users.  I also haven't checked href's within the math to see if they are accessible.)

Localization of the "Close Window" string that is used on the close buttons (though I'm wondering if this would be better as "Close Dialog" or "Close Pop-up" since it is not actually the window that is being closed).

A fix that makes sure aria- and role attributes show up as actual attributes in the DOM.

A fix to HTML-CSS so that the focus highlighting will be full width when the math is full width (e.g., when there is a \tag, or with percentage-width tables, or with the multline environment).

Fixes for handling of keyboard and menus in IE < 9.

Fix so that SVG elements won't be focusable in IE.

Fix Error jax so that [Math Processing Error] elements work with keyboard menus.

I added a few more fixes based on the results of running the test suite.

Fixed Error jax so that [Math Processing Error] elements work with the keyboard menus.

I think WCAG requests that focus should be visible. But I really don't mind.

I think WCAG requests that focus should be visible.

Note that the focus will be visible when the keyboard is used.  It is just if the menu or dialog is opened with the mouse that the focus is not visible.  This is is consistent with the example linked from the aria practices documentation.  When the mouse is used to open and select the menu items, the focus doesn't show, but when keyboard is used, it does.  This seems consistent with normal practice.  I've never seen menus that show focus outlines for mouse actions, or buttons that focus when you click them.  Perhaps it differs by operating system, but it seems very strange to me to see the focus outlines during mouse activity.

Localization of the "Close Window" string that is used on the close buttons (though I'm wondering if this would be better as "Close Dialog" or "Close Pop-up" since it is not actually the window that is being closed).

I also would prefer close dialog (or close alert). However, I am we should keep in mind that the Aria label is what the AT will read out in lieu of a button label. So what is more natural for people: window or dialog? Does your casual user even know what a dialog is?

So what is more natural for people: window or dialog? Does your casual user even know what a dialog is?

I really don't know.  But I was worried that "close window" might suggest that it is the browser window that is being closed, not that a pseudo-window is being closed within the browser window.  A visually impaired user might not really know that a window-like item is open.  Do we need to use the aria dialog attributes and techniques (see making dialogs modal, alert dialogs, and modal dialogs for instance)?

OK, I've resolved the conflicts.

Good point. So maybe close dialog is the right way to go. Or, since we are not using a generic close button we might as well be more specific and say "close help dialog" and  "close about mathjax dialog".
I believe we pretty much do everything re dialog/alert with the exception of using the document role for the message. And I am not sure that is really necessary given the tabbing order.

Should be good to merge now. It should also not clash with #1267.


I need more than 100 revisions in history.

Hi, I want to see all revisions in 'Source Control' -> 'History'.
100 revisions is sometimes not enough.
If it could hurts speed of getting history from SVC, making it with options could be OK.
Thanks.

Yes, as said previously, this could be increased, or made configurable, but this is not a priority for me right now


Independent States

Closes #3204

i think we need #2429 to solve this properly. the _was_touched marker is reset for every event (in redis), just because a particular substate wasn't touched, doesn't necessarily mean that it doesn't have computed vars that depend on vars in the parent states that may have changed.

@masenf thanks for looking into this. While my issue description with the unused page looks like it is related to #2429 it really isn't. This is a completely different issue. Maybe my last commit makes things more clear?

If my assumptions are correct, this should improve event handling performance for reflex-web quite decent. There are over 20 ComputedVars in reflex-web, even one which establishes a database connection. With reflex's current behavior, this should cause every event to recompute all those ComputedVars.

import reflex as rx

class StateA(rx.State):
    a: str = ""

    async def handle_click(self):
        self.a = "foo"
        state_b = await self.get_state(StateB)
        state_b.b = "bar"

class StateB(rx.State):
    b: str = ""

def index() -> rx.Component:
    return rx.vstack(
        rx.hstack(
            StateA.a,
            StateB.b
        ),
        rx.button("Click me", on_click=StateA.handle_click)
    )

app = rx.App()
app.add_page(index)

In reflex main branch, the page displays "foobar" after clicking the button.
With this proposed patch, the page only displays "foo".
This is primarily why the delta is calculated from the root state, so that any states which are patched into the tree during processing of an event have their deltas also calculated.
I definitely see what you're trying to do here; ComputedVar is re-calculated in a somewhat overzealous manner, but the reason for that is to retain the semantics of ensuring they always have the latest value after every event. My main motivation for introducing the @rx.cached_var decorator last year, was to avoid the cost of recomputing these vars when the vars they depend on have not been changed.
If I just have an @rx.var decorated function, reflex has no way of knowing what it depends on. Since it's in a state class that is ultimately a descendent of rx.State, it might depend on router vars or something else that could change when an event is processed. I don't see a way around recomputing it without opting in to @rx.cached_var semantics... so maybe it's time to make @rx.var be a cached var by default?

Thanks, I completely missed get_state. I already thought about the changing router vars...
Shouldn't it be possible to extend the computed var dependency tracking to respect get_state?

Shouldn't it be possible to extend the computed var dependency tracking to respect get_state?

Thankfully because computed vars use property semantics, they can only access vars in their immediate ancestors (without using hacks that we do not support). So the dependency tracking is already working i think. The problem is that for regular @rx.var we don't do any dependency tracking; an @rx.var is always considered dirty by definition.

Thanks a lot, I should have looked into the dependency tracking implementation earlier...
If the tracking works fine, is there any reason not promoting it to the default?

If the tracking works fine, is there any reason not promoting it to the default?

There have been several bugs initially in the dependency tracking that have since been fixed. It's possible that other bugs could be lurking, but i have not uncovered them yet and use @rx.cached_var in all of my projects. Initially the intent was to have cached var be the default behavior, but we were hesitant to bring it in at first without some more bake time.
I think it's ready to become the default, but that means we're relying on users to report bugs that might pop up related to computed vars in the next release.
Additionally, it notably cannot track vars outside of self, so if someone is using a computed var to access/render data that originates from outside the state, a cached_var will essentially never update in that case.

I see, makes sense. Considering the possible performance impacts of @rx.var I would support changing the default to _cache = True. I somehow was convinced that @rx.vars only should get updated if the associated state is dirty. But as you pointed out nicely, it's always considered dirty if it has an @rx.var defined.
I just migrated some codebases to cached var. If any problems arise, I will report them as usual.


Token Acess Page needs to be localized

Many non-english Korean users are having hard time.
this user think element tab is KC3 viewer.
Also, korean users use term "Token Link" instead of "API link" so they are pritty confused.
Sorry for bad english tho

html itself with language button could solve it

hm, sorry, not sure what you meant by "html itself with language button"?

ㄴ like this :

Could more easy to make then adding all those lines to translation file

oh I see what you mean. thanks. though will probably only after event

had browsed through source, but just adding translation file will be good.

well, as the [Token toggle disabled] is translate-able, why not for the whole page?
looks fun to make it happen.

bumping this after #825


chore: global update

update structure and contents for a first v2 beta release on production following our Release Early and Often principle.
WIP: several index pages need more content (especially computation and data capture), last steps tutorial still needs to be completed, and some old vocabulary still needs to be updated on the encryption and references sections

As we need to release this ASAP on https://next.blindnet.dev/, this PR is only for tracking purposes and will be directly merged without review.
See #31 for a quick review before release to production, and for retrospective review after release to production.


ESMValTool installed in development mode does not run

Describe the bug
After installing ESMValTool in development mode from scratch, running the tool with esmvaltool run recipe_python.yml fails immediately with a
> esmvaltool run recipe_python.yml
ERROR:esmvalcore._main:Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/home/manuel/ESMValCore/esmvalcore/_main.py", line 433, in run
    fire.Fire(ESMValTool())
  File "/home/manuel/miniconda3/envs/esmtest/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/manuel/miniconda3/envs/esmtest/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/manuel/miniconda3/envs/esmtest/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/manuel/ESMValCore/esmvalcore/_main.py", line 357, in run
    from ._recipe import TASKSEP
  File "/home/manuel/ESMValCore/esmvalcore/_recipe.py", line 11, in <module>
    from netCDF4 import Dataset
  File "/home/manuel/miniconda3/envs/esmtest/lib/python3.9/site-packages/netCDF4/__init__.py", line 3, in <module>
    from ._netCDF4 import *
ImportError: libmpi.so.12: cannot open shared object file: No such file or directory

The installation itself works well and doesn't raise an error. Here is the log of it: install.log
Doing a import iris or import netCDF4 in a python shell results in the same error.
Here is my conda environment: conda.log
Please note that immediately after environment creation with conda env create ... the import iris still works fine, so I guess the culprit is a something that is installed by pip. I tested this independently on two machines (DKRZ and my local laptop) and could reproduce it on both.
By the way do we have a test that actually runs `esmvaltool``? I couldn't find a failing test on the CI machines.
@ESMValGroup/esmvaltool-coreteam we should fix this before the release.

yes, we have the most basic one, esmvaltool help :grin: If you do a which pip what do you get Manu? I have noticed recently that conda decides to supersede the environment's pip with the one installed in base env and was not sure if that is an issue just on my machine; I'll have a look at this later on today :+1:

I did run into the same issue as @schlunma.
I think it is a bug in the netcdf4 conda-forge package:
Running mamba list netcdf4 or conda list netcdf4 in the environment returned a Build which starts with nompi_py39... so it should not depend on mpi, but
ldd $CONDA_PREFIX/lib/python3.9/site-packages/netCDF4/_netCDF4.cpython-39-x86_64-linux-gnu.so

lists a libmpi as not found library.
I was able to work around that with a simple conda update netcdf4, but clearly that's sub-optimal.
Two ways to look into this:

File an issue over at https://github.com/conda-forge/netcdf4-feedstock/issues (a problem may be that I think we are not on the most recent version and I don't know if an older version would be rebuild anyway).
Investigate where exactly the pinning is coming from and try to move this to a newer version. Particularly, have a look at effects of the mpich pinning.

OK I have just created a brand new env (with mamba since conda would not finish even after 30min heh) and I believe mpich is the problem here, you have a dodgy floater off external_0 build (always, always stay away from external builds), if I update my mpich to that I run in exactly the same lib import issue (note that --help actually works, but a test run doesn't, but we do have test runs in the CI from the test_recipe, it's just that the CI doesn't pick up that mpich version). Note that I have exactly the same netcdf4 and pip etc as you - mpich is again the culprit (we were pinning on it until recently, I need to reintroduce the pin). Working version:
# Name                    Version                   Build  Channel
mpich                     3.3.2                h846660c_5    conda-forge

mpich has a linux-64 native build 3.4.2 as we can see in the package list - why conda/mamba want to install the external build is beyond me

mpich has a linux-64 native build 3.4.2 as we can see in the package list - why conda/mamba want to install the external build is beyond me

trying to force that at env solving while asking for the latest mpich with "mpich[version='>=3.4.2,!=3.4.2',build=external_0]" results in env solving error, I guess there is a conflicter package that really wants that or below. Instead of doing that, pinning netcdf4>=1.5.7 results in package conflict as well:
Encountered problems while solving:
  - package ncl-6.6.2-hc2f75e0_2 requires libgdal >=2.4.1,<2.5.0a0, but none of the providers can be installed

We'll have to pin mpich to <3.4 for now

just noticed that we do indeed pin mpich<3.4 in the CI config meta.yml (I must have left it there inadvertently when I removed the pin in the other two files gah)

Possibly relevant: https://github.com/conda-forge/mpich-feedstock/issues/55

yep that's the bugger @zklaus - it appears there are possible solutions but nothing solid, well, more solid than pinning to an older version. Let's keep an eye on this stuff, maybe we should open an issue and close it only when we can get a definitive solution?

Yeah, maybe we can spitball some ideas there. For example, could it be possible to forbid *external* builds?

Cheers @valeriupredoi and @zklaus, thanks for the quick fix and sorry for the slow response! Just re-tested this and everthing works well now! :tada:
Just one quick question: Do we need a similar treatment for ESMValCore's environment.yml and meta.yml?


One drive for business api retun less number of records

I m using OneDrive for Business API to get folders and files inside a folder like this
/_api/v2.0/me/drive/items/01ZFGAWKSQXL************7Y/children
which is working fine.
But it is not returning all records but only few. I have more than 3000 folders but it is returning only 200
Here
https://dev.onedrive.com/items/list.htm
I found that there is a default limit of 200 items, and  @odata.nextLink property can be use to get next set of records. And for this i have to create a RECURSIVE method. But to get 3000 records in set of 200 it will make approx 16 API calls.
Please suggest is the a right way to get all records or is there any better approach for it.

The OneDrive API implements standard paging semantics to protect both the clients and the service from unbounded requests. As a result, for a sufficiently large folder making N calls is unavoidable.
You can minimize the number of round trips by specifying the maximum page size, which I believe is 1000:
/_api/v2.0/drive/items/ABCD/children?top=1000

Thanks @ificator for your suggestion

Hi @ificator
i want to search a file or folder based on its name only.
It is mentioned that The query text used to search for items. Values may be matched across several fields including filename, metadata, and file content.
I used filter as well like this
/_api/v2.0/me/drive/items/01Z54FA3E**********2XOG2FQOJ5/search(q='u')?filter=contains(name,'u')
but it still return other items as well.
Is it possible to search based on name only?


Documentation page for Scheduled Queries

We will likely need a new page here https://help.plot.ly/database-connectors/ to walk users through the new Scheduled Queries pane and workflow. /cc @cldougl

👍 -- moving over to the help doc repo to add to the queue: https://github.com/plotly/plotly.github.io/issues/358


Allow specifications to use spec or doap namespaces

It already supported spec:Specification so the changes were to make the ReportGenerator look for both and to change the test files to replace some spec: examples with doap:


Greenkeeper/enzyme to json 3.1.4

Close #17

Coverage remained the same at 100.0% when pulling e8075179db1aedc5c915b0206ef4dfbff16d58cd on greenkeeper/enzyme-to-json-3.1.4 into 28396bd040d7e6dd3bf08d88a1b1d69d7e350094 on master.


make PR #1193 more compact and readable

@jhmigueles with this revision the code is more compact and readable.

Checklist before merging:

[ ] Existing tests still work (check by running the test suite, e.g. from RStudio).
[ ] Added tests (if you added functionality) or fixed existing test (if you fixed a bug).
[ ] Clean code has been attempted, e.g. intuitive object names and no code redundancy.
[ ] Documentation updated:

[ ] Function documentation
[ ] Chapter vignettes for GitHub IO
[ ] Vignettes for CRAN

[ ] Corresponding issue tagged in PR message. If no issue exist, please create an issue and tag it.
[ ] Updated release notes in inst/NEWS.Rd with a user-readable summary. Please, include references to relevant issues or PR discussions.
[ ] Added your name to the contributors lists in the DESCRIPTION file, if you think you made a significant contribution.
[ ] GGIR parameters were added/removed. If yes, please also complete checklist below.

Thank you @vincentvanhees. I have just tested this and it works well in my files.
I provided more information in the issue, and additionally you proposed revising the NEWS details. Would something like this be clearer?

Part 5: Revised minor bug when incorporating the sleep period time windows into part 5 time series in g.part5.wakesleepwindows. If the timestamp needs to be rounded, e.g., "9:16:58" for an epochSize of 5 would result in "9:16:60", yet this would not be matched with the time series and it would result in a misclassification of the sleep period time. Now, this is converted to "09:17:00" to correctly match it with the time series. #1192

Thanks, I will merge this PR now and then will have a look at NEWS item in the main PR.


Save the API rate limit

Hi! Thank you a lot for your library.
It would be interesting to save the API rate limit and expose it.
Idea for the implementation:

node-strava-v3, rateLimiting.js
node-strava-v3, httpClient.js

What do you think of the feature? What do you think of the node-strava-v3's interface?
I guess you prefer if I submit a PR?

Feel free to submit a PR, I think it can be useful to expose these.


add configuration file for textured meshes

allows for pose selector visualizer to show detected objects in rviz in color

I've just pushed a commit that does the same thing a little differently, so this PR can be closed: 90ae4f64


Load Data into Standalone Data Lake in SAP HANA Cloud

Tutorials: https://developers.sap.com/tutorials/hana-cloud-hdl-getting-started-3.html
In Step 2 (the first SQL Code block in Step 2), there's an extra '*/' at the end of line 1. This causes the SQL to not work correctly and must be removed manually after it is copied

Hi,
Thank you for letting us know. This error will be corrected soon. Hope the tutorial was helpful.
Regards,
Vijay

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.


Fix delay when calling vim-picker mappings

See https://github.com/srstevenson/vim-picker/issues/50.

Cool, thank you for being proactive @srstevenson .


2727 global analytics 2

Replacing PR https://github.com/ministryofjustice/fb-editor/pull/1618 for better history.
https://trello.com/c/h3595yEp/2727-show-cookie-banner-when-global-analytics-is-configured-for-a-form
Use latest Metadata Presenter 2.17.9
(Workaround) Fix for JS errors when in Preview

The cookie banner will be shown in preview for a form, even when there are no measurement IDs present. Is this the intended behaviour?

The cookie banner will be shown in preview for a form, even when there are no measurement IDs present. Is this the intended behaviour?

That's the intended behaviour on Runner. Preview in Editor is a more interesting question. With the Cookie banner showing is a true representation of the form view, at least until the user makes a choice. I would listen to arguments that it is not necessary to show in just a preview, however. I guess the correct way to handle it would be to have an environment switch that knows it is Preview and update the show_banner method to logic along the lines of 'if not preview and there is no banner cookie to indicate a user choice' show the banner. We could also, put in a workaround of just using CSS to hide it using the Editor stylesheets. What do you think?


DOM interaction

I just tried this out and am fairly impressed! Just wondering, is it possible yet to interact with the DOM from my ruby script?

Okay, so I managed to at least execute JS code from ruby (without getting any return value though).
I extended assets/mruby_init.c to provide a global ruby method run_js that uses the emscripten API to run JS code in the browser:
#include <mruby.h>
#include <mruby/irep.h>
#include <emscripten.h>

mrb_value run_js(mrb_state* mrb, mrb_value self)
{
  char *js_code;
  mrb_get_args(mrb, "z", &js_code);
  emscripten_run_script(js_code);
  return self;
}

int main() {
  mrb_state *mrb = mrb_open();

  if (!mrb) { /* handle error */ }

  mrb_define_method(mrb, mrb->kernel_module, "run_js", run_js, MRB_ARGS_REQ(1));

  mrb_load_irep(mrb, ruby_app);

  // If an exception, print error
  if (mrb->exc) mrb_print_error(mrb);

  mrb_close(mrb);
  return 0;
}

When compiling the following hello.rb to wasm, the browser shows the alert.
run_js("window.alert(\"foo\");")

I don't know whether this is a good approach or if it was even possible in another, more straightforward way. But if not, this proof of concept could be encapsulated into a mrbgem that defines some API to interact really in both directions with the DOM in the browser.

Could this be possible using this crate?
https://github.com/web-dom/web-dom


As a user, I want to be able to report problems to help improve the challenges.

Acceptance Criteria

[x] User can easily find the “report a problem” button on any screen of the application.
[x] User can easily fill out a title and description to submit with any logs / metrics from the application.

This is performed by the qualtrics survey, which will be expanded upon as more information is required.


Fix eth gas limit missing

@4gn3s I pulled your changes and fixed the tests

Coverage increased (+1.03%) to 52.044% when pulling e2a028adb9fef3d9d53f6185bba70585413448bf on RomanZacharia:fix_eth_gasLimit_missing into dc47ef8da6f93aa851503deccb529c0a281d2529 on ethereum:develop.

Addresses https://github.com/ethereum/pyethapp/issues/150


[New Project]: DHIS2 UI Library

Is this project already existing in the app?

[X] I have searched the existing projects

Tell us about the project
https://github.com/dhis2/ui
DHIS2 is the world's largest open-source health information management system. Another project from the organisation is already listed (dhis2-core), but not this one. This is the UI library used for building  DHIS2 applications and we want to start exposing some of the issues we want to tackle there for help from the global community.
Is the Project a DPG (Digital Public Good)

[X] Yes
[ ] No

Does the project list which Sustainable Development Goals it addresses in its labels?

[X] Yes
[ ] No

Code of Conduct

[X] I've read the Code of Conduct and understand my responsibilities as a member of the GitHub community

Anything else?
The project is a DPG, but I can't submit the form unless I choose No as an answer

Added via your PR! :)


basic.ds sample spec fails under C++ flag

...where it works for --java.
$ cozy basic.ds --c++ -
...
[('_var10806', TNativeMap(TInt(), TNativeList(TInt()))), ('_var11144', TNativeMap(TInt(), TBool())), ('_var894', TNativeMap(TInt(), TNativeList(TInt()))), ('_var12', TNativeList(TInt()))]
Code generation failed!
Implementation was dumped to /tmp/failed_codegen.py
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/bin/cozy", line 11, in <module>
    load_entry_point('Cozy', 'console_scripts', 'cozy')()
  File "/Users/dgrant/Dev/cozy/cozy/cozy/main.py", line 95, in run
    out.write(compile.CxxPrinter().visit(impl, state_map, share_info))
  File "/Users/dgrant/Dev/cozy/cozy/cozy/common.py", line 141, in visit
    return getattr(self, visit_func)(x, *args, **kwargs)
  File "/Users/dgrant/Dev/cozy/cozy/cozy/compile.py", line 694, in visit_Spec
    for (name, t) in spec.statevars))
  File "/Users/dgrant/Dev/cozy/cozy/cozy/compile.py", line 694, in <genexpr>
    for (name, t) in spec.statevars))
  File "/Users/dgrant/Dev/cozy/cozy/cozy/compile.py", line 660, in initial_value
    return self.initial_value(t.rep_type())
AttributeError: 'TNativeList' object has no attribute 'rep_type'

(Same for in.ds, map.ds, nested-map.ds.)

Fixed by de8cbe016a64fe91773d9badff3f1eb530ed8c00.

However, the generated code still does not compile thanks to #4


Organize test suite into classes

Currently tests are organized into modules and then separated into sections using comment rules e.g.
# pin_download ===================================================================
This can make it hard to navigate the test suite and also results in longer test names: the test names include both the category information and the specifics of the test.
Pytest allows you to organize tests into classes. Besides adding some clarity to the test organization, this also allows the user to have more fine-grained control about which tests run.
So I would propose changing the following:
# pin_write ===================================================================

def test_board_pin_write_default_title(board):

    df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})
    meta = board.pin_write(df, "df_csv", title=None, type="csv")
    assert meta.title == "df_csv: a pinned 3 x 2 DataFrame"

def test_board_pin_write_prepare_pin(board, tmp_dir2):

    df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})

    meta = board.prepare_pin_version(
        str(tmp_dir2), df, "df_csv", title=None, type="csv"
    )
    assert meta.file == "df_csv.csv"
    assert (tmp_dir2 / "data.txt").exists()
    assert (tmp_dir2 / "df_csv.csv").exists()
    assert not (tmp_dir2 / "df_csv.csv").is_dir()

def test_board_pin_write_roundtrip(board):

    df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})

    assert not board.pin_exists("df_csv")

    board.pin_write(df, "df_csv", type="csv")

    assert board.pin_exists("df_csv")

    loaded_df = board.pin_read("df_csv")
    assert loaded_df.equals(df)

def test_board_pin_write_type_not_specified_error(board):
    class C:
        pass

    with pytest.raises(NotImplementedError):
        board.pin_write(C(), "cool_pin")

To this:
class TestBoard:
    ... # All the other sets of tests on pins.board.BaseBoard methods...
    class TestPinWrite:
        def test_default_title(self, board):

            df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})
            meta = board.pin_write(df, "df_csv", title=None, type="csv")
            assert meta.title == "df_csv: a pinned 3 x 2 DataFrame"

        def test_prepare_pin(self, board, tmp_dir2):

            df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})

            meta = board.prepare_pin_version(
                str(tmp_dir2), df, "df_csv", title=None, type="csv"
            )
            assert meta.file == "df_csv.csv"
            assert (tmp_dir2 / "data.txt").exists()
            assert (tmp_dir2 / "df_csv.csv").exists()
            assert not (tmp_dir2 / "df_csv.csv").is_dir()

        def test_roundtrip(self, board):

            df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})

            assert not board.pin_exists("df_csv")

            board.pin_write(df, "df_csv", type="csv")

            assert board.pin_exists("df_csv")

            loaded_df = board.pin_read("df_csv")
            assert loaded_df.equals(df)

        def test_type_not_specified_error(self, board):
            class C:
                pass

            with pytest.raises(NotImplementedError):
                board.pin_write(C(), "cool_pin")

        ... # etc.

My plan for making this change would be one test file at a time.

The # pin_write ========== is probably a good signal that some of the tests would be better organized into classes. I don't anticipate all the tests make sense as classes, but boards/caches seem like good places to start.


CASMTRIAGE-6296: add a reference for complete shutdown sequence

Description
When performing a complete system shutdown, if the admin starts with management cluster shutdown, it makes it very difficult to shut down the rest of nodes. This PR adds a reminder to follow the expected shutdown sequence.

Checklist

[ ] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[ ] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[x] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.4,release/1.6

/backport release/1.4 release/1.6


chore(merge): merge release/v2.0.0 to main

Description
This merges the release/v2.0.0 release branch into main.
 Fix expected to be in place by the end of the week. Might need to disable visual snapshot testing for now to unblock PRs.


Plugin not working

Hello.
I have Domoticz 2020.2, installed everything, but the plugin not working. In the Domoticz log I have:
'onMessage' failed 'TypeError':'onMessage() missing 2 required positional arguments: 'Connection' and 'Data''.
When I try to use the get_dps.py, I have:
Unexpected status() payload=b'\xb3\xc9\x80\xfd\x8a5a\xf1DB\xa2\xb8U\xf6\x08k\xbf\xab\xd6\x9bZ\xecO\xc7)\xb7\xb6}*\x02\xf5 '
Plug State Information:
b'\xb3\xc9\x80\xfd\x8a5a\xf1DB\xa2\xb8U\xf6\x08k\xbf\xab\xd6\x9bZ\xecO\xc7)\xb7\xb6}*\x02\xf5 '
Plug DPS List:
Traceback (most recent call last):
File "./get_dps.py", line 58, in
for key in data['dps'].keys():
TypeError: byte indices must be integers or slices, not str
I have the key etc, but something goes wrong :( . Help please.
Thanks

I found that the problem is with pytuya! Can we move to tinytuya: https://github.com/jasonacox/tinytuya
Which is working fine!
Also have a look to this fork: https://github.com/sincze/Domoticz-Tuya-SmartPlug-Plugin
which adds sensors support.
Now we have everything to make this plugin to work fine, can you implement it?
Thanks


Fix: InmemoryDB minor fixes

What does this fix or implement?

Checklist

[ ] PR name added as appropriate (e.g. feat:/fix:/doc:/test:/refactor:)
[ ] Tests added or updated
[ ] Documentation updated
[ ] Changelog updated and version incremented (label: upcoming release)
[ ] Github Issue linked if any
[ ] Jira task updated

@cbeti-ionos forgot to mention before approving, please update the CHANGELOG.md


Delete multiple Chips

After deleting first chip . How can i refresh chips . Next time  i select first chip again it gives me position 1 instead of zero

You've found a bug, I'll have a fix out some time this week.

how do you delete a chip from chipCloud ? I can't find it any method for it

Sorry, I've not had time to look into this yet. The library wasn't intended to handle this use-case, but would be good to have it.

Use this property.
chipCloud.removeViewAt(indexPosition);

@fiskurgit It's all right I thought I missed it. I'll fork and add it and submit pull request or you can cherry-pick.
@davin12x  I tried that but my app crashes and android studio doesn't catch it in its log too

Sorry this was left for so long. The bug was because the chips hold their index internally, so when you delete one in the middle of the array they become out of sync, solution was to reinitialise the indexes when one is removed.
Working code to delete a selected chips as example:
final ChipCloud chipCloud = (ChipCloud) findViewById(R.id.chip_cloud_a);
String[] demoLabels = new String[]{"UY Scuti", "WOH G64", "RW Cephei", "Westerlund 1-26", "V354 Cephei", "KY Cygni", "VY Canis Majoris"};
chipCloud.addChips(demoLabels);

Button deleteButton = (Button) findViewById(R.id.delete_selected_button);
deleteButton.setOnClickListener(new View.OnClickListener() {
  @Override public void onClick(View view) {
    int selectedIndex = chipCloud.getSelectedIndex();
    chipCloud.deleteChip(selectedIndex);
  }
});

getSelectedIndex() and deleteChip(int index) methods available in 2.2.1

deleteChip is no longer an available method, don't see any way to delete a chip(s)


When loading new theme

While waiting for the image of the next theme, an animated border with the gradient from your twitch cam view can appear around the old image( or instead of it) so users can see something is happening

I mean something like this. And it could be an idea to disable all the links until new theme is loaded. ( also in css [cursor: not-allowed;] or something like it )
https://user-images.githubusercontent.com/10188317/116789699-5e6b7a00-aab0-11eb-904c-cfc3bfe29981.mp4


Kerberos - javax.security.sasl.SaslException. Client mechanism is malformed

I am using snakebite with kerberos.
When I do snakebite -D ls /, I notice this error:
status: FATAL
serverIpcVersionNum: 9
exceptionClassName: "javax.security.sasl.SaslException"
errorMsg: "Client mechanism is malformed"
errorDetail: FATAL_UNAUTHORIZED
clientId: ""
retryCount: -1

I have cyrus-sasl pkg installed. Prior to this failure, snakebite encountered 2 mechanisms - DIGEST-MD5 and GSSAPI. But I am not sure if it chose the right one because the log for chosen mech is empty
DEBUG:snakebite.channel:Response:
state: NEGOTIATE
auths {
  method: "TOKEN"
  mechanism: "DIGEST-MD5"
  protocol: ""
  serverId: "default"
  challenge: "some challenge callback"
}
auths {
  method: "KERBEROS"
  mechanism: "GSSAPI"
  protocol: "hdfs"
  serverId: "some server"
}

DEBUG:snakebite.rpc_sasl:Available mechs: DIGEST-MD5,GSSAPI
DEBUG:snakebite.rpc_sasl:Chosen mech:
DEBUG:snakebite.channel:Sending: 00 00 00 10 (len: 4)

Not sure if I am missing a package or there is a bug in the negotiation or something else
cc: @bolkedebruin

So you have a valid ticket? Show the output of klist
Sent from my iPhone

On 8 okt. 2015, at 21:36, praveev<EMAIL_ADDRESS>wrote:
I am using snakebite with kerberos.
When I do snakebite -D ls /, I notice this error:
status: FATAL
serverIpcVersionNum: 9
exceptionClassName: "javax.security.sasl.SaslException"
errorMsg: "Client mechanism is malformed"
errorDetail: FATAL_UNAUTHORIZED
clientId: ""
retryCount: -1
I have cyrus-sasl pkg installed. Prior to this failure, snakebite encountered 2 mechanisms - DIGEST-MD5 and GSSAPI. But I am not sure if it chose the right one because the log for chosen mech is empty
DEBUG:snakebite.channel:Response:
state: NEGOTIATE
auths {
method: "TOKEN"
mechanism: "DIGEST-MD5"
protocol: ""
serverId: "default"
challenge: "some challenge callback"
}
auths {
method: "KERBEROS"
mechanism: "GSSAPI"
protocol: "hdfs"
serverId: "some server"
}
DEBUG:snakebite.rpc_sasl:Available mechs: DIGEST-MD5,GSSAPI
DEBUG:snakebite.rpc_sasl:Chosen mech:
DEBUG:snakebite.channel:Sending: 00 00 00 10 (len: 4)
Not sure if I am missing a package or there is a bug in the negotiation or something else
cc: @bolkedebruin
—
Reply to this email directly or view it on GitHub.

klist output
Ticket cache: FILE:/tmp/krb5cc_0
Default principal<EMAIL_ADDRESS>
Valid starting     Expires            Service principal
10/08/15 20:11:01  10/09/15 20:11:01<EMAIL_ADDRESS>	renew until 10/15/15 20:11:01

sasl chooses that mech that in can support. I am only getting this output when the ticket has expired.
expired ticket, but available ticket cache:
snakebite -D ls /
DEBUG:snakebite.rpc_sasl:Available mechs: DIGEST-MD5,GSSAPI
DEBUG:snakebite.rpc_sasl:Chosen mech:

destroy
snakebite ls /
krbV.Krb5Error: (-1765328189, 'No credentials cache found')

kinit bolke
Password for<EMAIL_ADDRESS>
snakebite ls /
Found 7 items
drwxrwxrwx   - yarn       hadoop              0 2015-08-21 10:08 /app-logs
drwxr-xr-x   - hdfs       hadoop              0 2015-08-21 10:07 /apps
drwxr-xr-x   - hdfs       hadoop              0 2015-08-21 10:06 /hdp
drwxr-xr-x   - mapred     hdfs                0 2015-08-21 10:06 /mapred
drwxrwxrwx   - mapred     hadoop              0 2015-08-21 10:06 /mr-history
drwxrwxrwx   - hdfs       hdfs                0 2015-08-21 10:10 /tmp
drwxr-xr-x   - hdfs       hdfs                0 2015-08-21 10:07 /user

so try kdestroy & kinit

hmm....yea tried kdestroy and kinit.
kdestroy does what it was supposed to.
kinit with keytab worked. But snakebite ls /  err'd out again with Client mechanism is malformed
This shouldn't matter but does it make a difference if I use a keytab to kinit instead of a straight up password?

Can you check if this still works for you if you use a keytab file instead?
For whatever reason, I have duplicate keys in the keytab file (with the same principal and kvno). Wonder if that could be an issue.

Might be. In my environment it works fine with a key tab (just tested it). Do other utilities work? ie. hadoop command line?

hmm.....ok. yea. my hadoop command line works fine.

Ok. To be honest I am a bit lost, because I cannot reproduce it except with a expired ticket in a cache. Furthermore, not selecting a mech is dependent on the underlying sasl (system) libraries. What you could try is shoot a bit broader and create a new user principal and try using that to connect and/or generate a new key tab.
If you you share your outputs then please include everything (ie full command lines and debug output).

Here is the complete output. I don't own the hadoop or kerberos systems I am working with. I am trying to setup a small dev env. to debug better.
[praveev@flame03 ~]$ /usr/bin/kinit -V -k -t /home/praveev/praveev.prod.headless.keytab praveev
Using default cache: /tmp/krb5cc_90677
Using principal<EMAIL_ADDRESS>Using keytab: /home/praveev/praveev.keytab
Authenticated to Kerberos v5

[praveev@flame03 ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_90677
Default principal<EMAIL_ADDRESS>
Valid starting     Expires            Service principal
10/08/15 15:36:52  10/09/15 15:36:52<EMAIL_ADDRESS>	renew until 10/15/15 15:36:52

[praveev@flame03 ~]$ snakebite -D ls /
DEBUG:snakebite.config:Got namenode 'hdfs://master-nn:8020/' from /home/praveev/hadoop/conf/core-site.xml
DEBUG:snakebite.config:Got hadoop.security.authentication 'kerberos'
DEBUG:snakebite.config:hdfs principal found<EMAIL_ADDRESS>DEBUG:snakebite.client:Switch to namenode: master-nn:8020
DEBUG:snakebite.client:Created client for master-nn:8020 with trash=False and sasl=True
DEBUG:snakebite.client:Trying to find path /
DEBUG:snakebite.channel:############## CONNECTING ##############
DEBUG:snakebite.channel:Sending: 68 72 70 63 (len: 4)
DEBUG:snakebite.channel:Sending: 09 (len: 1)
DEBUG:snakebite.channel:Sending: 00 (len: 1)
DEBUG:snakebite.channel:Sending: df (len: 1)
DEBUG:snakebite.channel:Sending: 00 00 00 0e (len: 4)
DEBUG:snakebite.channel:Sending: 0a (len: 1)
DEBUG:snakebite.channel:Sending: 08 02 10 00 18 41 22 00 28 01 (len: 10)
DEBUG:snakebite.channel:Sending: 02 (len: 1)
DEBUG:snakebite.channel:Sending: 10 01 (len: 2)
DEBUG:snakebite.rpc_sasl:Send out:

state: NEGOTIATE

DEBUG:snakebite.channel:############## RECVING ##############
DEBUG:snakebite.channel:############## PARSING ##############
DEBUG:snakebite.channel:Payload class: <class 'snakebite.protobuf.RpcHeader_pb2.RpcSaslProto'>
DEBUG:snakebite.channel:Bytes read: 4, total: 4
DEBUG:snakebite.channel:Total response length: 226
DEBUG:snakebite.channel:Bytes read: 4, total: 8
DEBUG:snakebite.channel:Delimited message length (pos 1): 14
DEBUG:snakebite.channel:Rewinding pos 7 with 3 places
DEBUG:snakebite.channel:Reset buffer to pos 4
DEBUG:snakebite.channel:Bytes read: 11, total: 19
DEBUG:snakebite.channel:Delimited message bytes (14): 08 df ff ff ff 0f 10 00 18 09 3a 00 40 01 (len: 14)
DEBUG:snakebite.channel:Header read 15
DEBUG:snakebite.channel:RpcResponseHeaderProto:

callId:<PHONE_NUMBER>
status: SUCCESS
serverIpcVersionNum: 9
clientId: ""
retryCount: -1

DEBUG:snakebite.channel:header: 15, total: 226
DEBUG:snakebite.channel:Bytes read: 211, total: 230
DEBUG:snakebite.channel:Delimited message length (pos 2): 209
DEBUG:snakebite.channel:Rewinding pos 229 with 209 places
DEBUG:snakebite.channel:Reset buffer to pos 20
DEBUG:snakebite.channel:Delimited message bytes (209): some bytes (len: 209)
DEBUG:snakebite.channel:Response:

state: NEGOTIATE
auths {
  method: "TOKEN"
  mechanism: "DIGEST-MD5"
  protocol: ""
  serverId: "default"
  challenge: "some-challenge"
}
auths {
  method: "KERBEROS"
  mechanism: "GSSAPI"
  protocol: "hdfs"
  serverId: "master-nn"
}

DEBUG:snakebite.rpc_sasl:Available mechs: DIGEST-MD5,GSSAPI
DEBUG:snakebite.rpc_sasl:Chosen mech:
DEBUG:snakebite.channel:Sending: 00 00 00 10 (len: 4)
DEBUG:snakebite.channel:Sending: 0a (len: 1)
DEBUG:snakebite.channel:Sending: 08 02 10 00 18 41 22 00 28 01 (len: 10)
DEBUG:snakebite.channel:Sending: 04 (len: 1)
DEBUG:snakebite.channel:Sending: 10 02 1a 00 (len: 4)
DEBUG:snakebite.rpc_sasl:Send out:

state: INITIATE
token: ""

DEBUG:snakebite.channel:############## RECVING ##############
DEBUG:snakebite.channel:############## PARSING ##############
DEBUG:snakebite.channel:Payload class: <class 'snakebite.protobuf.RpcHeader_pb2.RpcSaslProto'>
DEBUG:snakebite.channel:Bytes read: 4, total: 4
DEBUG:snakebite.channel:Total response length: 83
DEBUG:snakebite.channel:Bytes read: 4, total: 8
DEBUG:snakebite.channel:Delimited message length (pos 1): 82
DEBUG:snakebite.channel:Rewinding pos 7 with 3 places
DEBUG:snakebite.channel:Reset buffer to pos 4
DEBUG:snakebite.channel:Bytes read: 79, total: 87
DEBUG:snakebite.channel:Delimited message bytes (82): 08 df ff ff ff 0f 10 02 18 09 22 21 6a 61 76 61 78 2e 73 65 63 75 72 69 74 79 2e 73 61 73 6c 2e 53 61 73 6c 45 78 63 65 70 74 69 6f 6e 2a 1d 43 6c 69 65 6e 74 20 6d 65 63 68 61 6e 69 73 6d 20 69 73 20 6d 61 6c 66 6f 72 6d 65 64 30 0f 3a 00 40 01 (len: 82)
DEBUG:snakebite.channel:Header read 83
DEBUG:snakebite.channel:RpcResponseHeaderProto:

callId:<PHONE_NUMBER>
status: FATAL
serverIpcVersionNum: 9
exceptionClassName: "javax.security.sasl.SaslException"
errorMsg: "Client mechanism is malformed"
errorDetail: FATAL_UNAUTHORIZED
clientId: ""
retryCount: -1

DEBUG:snakebite.client:Request failed with javax.security.sasl.SaslException
Client mechanism is malformed
Request error: javax.security.sasl.SaslException
Client mechanism is malformed
```

@bolkedebruin  making progress! I got it working with kerberos on the command line.
But it does not work from the code from airflow. I suspect it has something to do with the principal name in airflow.cfg.
Should the principal be in this format<EMAIL_ADDRESS>where _host will get resolved into the hostname?

Excellent! Can you share what did you do to get it to work to also help others?
For airflow: it should be the principal that you can access your environment with, ie your own. Besides that the principal gets augmented with the host (which it maybe should not do now I am thinking about it).

Op 9 okt. 2015, om 21:49 heeft praveev<EMAIL_ADDRESS>het volgende geschreven:
@bolkedebruin https://github.com/bolkedebruin making progress! I got it working with kerberos on the command line.
But it does not work from the code from airflow. I suspect it has something to do with the principal name in airflow.cfg.
Should the principal be in this format<EMAIL_ADDRESS>where _host will get resolved into the hostname?
—
Reply to this email directly or view it on GitHub https://github.com/spotify/snakebite/issues/175#issuecomment-146971863.

In hdfs-site.xml, I incorrectly had dfs.namenode.kerberos.principal to be praveev instead of<EMAIL_ADDRESS>Here is a copy of hdfs-site and core-site that worked for me.
hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration xmlns:xi="http://www.w3.org/2001/XInclude">

  <property>
    <name>dfs.namenode.kerberos.principal</name>
<EMAIL_ADDRESS>    <description>
        Kerberos principal name for the NameNode
    </description>
  </property>

</configuration>

core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration xmlns:xi="http://www.w3.org/2001/XInclude">

<xi:include href="/home/gs/conf/local/local-superuser-conf.xml" />
  <!-- file system properties -->

  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://uraniumblue-nn1.blue.ygrid.yahoo.com:8020</value>
    <description>The name of the default file system. Either the
      literal string "local" or a host:port for HDFS.</description>
    <final>false</final>
  </property>

  <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
    <description>
      Set the authentication for the cluster. Valid values are: simple or
      kerberos.
   </description>
  </property>

</configuration>

@bolkedebruin - for the airflow part, maybe we should take this conversation to airflow repo.
here's my kerberos related parts of airflow.cfg and this doesn't work:
[core]
security = kerberos
[kerberos]
ccache = /tmp/krb5cc_90677
principal = praveev
reinit_frequency = 3600
kinit_path = kinit
keytab = /home/praveev/praveev.keytab
In the command line, the principal is of the form<EMAIL_ADDRESS>where _HOST is my hostname. It doesn't use the default principal in the ticket.
My klist:
Ticket cache: FILE:/tmp/krb5cc_90677
Default principal<EMAIL_ADDRESS>Valid starting     Expires            Service principal
10/09/15 12:18:57  10/10/15 12:18:57<EMAIL_ADDRESS>renew until 10/16/15 12:18:57
10/09/15 12:19:25  10/10/15 12:18:57<EMAIL_ADDRESS>renew until 10/16/15 12:18:57
I got the service principal<EMAIL_ADDRESS>after doing the command line.

@bolkedebruin - for the airflow part, maybe we should take this conversation to airflow repo.
here's my kerberos related parts of airflow.cfg and this doesn't work:
[core]
security = kerberos
[kerberos]
ccache = /tmp/krb5cc_90677
principal = praveev
reinit_frequency = 3600
kinit_path = kinit
keytab = /home/praveev/praveev.keytab
For the command line, the principal is of the form<EMAIL_ADDRESS>where _HOST is my hostname. It doesn't use the default principal in the ticket.
For airflow, in airflow/security/kerberos.py principal is defined as
   principal = "%s/%s" % (conf.get('kerberos', 'principal'), socket.getfqdn()) which is not quite the same as the ones from command line b/c i don't see it include the @ part of it ( i.e @LOCAL.COM)
My klist
Ticket cache: FILE:/tmp/krb5cc_90677
Default principal<EMAIL_ADDRESS>Valid starting     Expires            Service principal
10/09/15 12:18:57  10/10/15 12:18:57<EMAIL_ADDRESS>renew until 10/16/15 12:18:57
10/09/15 12:19:25  10/10/15 12:18:57<EMAIL_ADDRESS>renew until 10/16/15 12:18:57
I got the service principal<EMAIL_ADDRESS>after executing from the command line.

@bolkedebruin I am unable to replicate the command line behavior with the library.
Script-1
from snakebite.client import Client
client = Client("hadoop-namenode", 8020, use_trash=True, use_sasl=True)
for x in client.ls(['/']):
....print x
Script-2
from snakebite.client import Client
client = Client("hadoop-namenode", 8020, use_trash=True, use_sasl=True<EMAIL_ADDRESS>for x in client.ls(['/']):
....print x
hadoop-namenode is the one that is there in core-site.xml
effective_user is the principal value from hdfs-site.xml

I think you need to read up a little how Kerberos works and how it works in a Hadoop environment.
For now I would leave out the effective user specifically because your are configuring it as the hdfs administrative user.
For the principal use the principal that you are using to do a kinit with.
And yes this thread should be on the airflow mailing list.

On 10 okt. 2015, at 01:08, praveev<EMAIL_ADDRESS>wrote:
@bolkedebruin I am unable to replicate the command line behavior with the library.
Script-1
from snakebite.client import Client
client = Client("hadoop-namenode", 8020, use_trash=True, use_sasl=True)
for x in client.ls(['/']):
....print x
Script-2
from snakebite.client import Client
client = Client("hadoop-namenode", 8020, use_trash=True, use_sasl=True<EMAIL_ADDRESS>for x in client.ls(['/']):
....print x
hadoop-namenode is the one that is there in core-site.xml
effective_user is the principal value from hdfs-site.xml
—
Reply to this email directly or view it on GitHub.

I'm having the same issue, though nothing to do with Airflow.
The CLI works fine
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_0)
root@ip-10-0-0-165:~# snakebite -v
2.7.2

root@ip-10-0-0-165:~# klist
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_0)

root@ip-10-0-0-165:~# snakebite ls /
FAILS as expected

root@ip-10-0-0-165:~# kinit edh
Password for edh@HADOOP:

root@ip-10-0-0-165:~# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: edh@HADOOP

Valid starting       Expires              Service principal
10/22/2015 22:20:11  10/23/2015 22:20:11  krbtgt/HADOOP@HADOOP
	renew until 10/29/2015 22:20:11

root@ip-10-0-0-165:~# snakebite ls /
Found 3 items
drwxrwxr-x   - solr       solr                0 2015-10-20 21:05 /solr
drwxrwxrwx   - hdfs       supergroup          0 2015-10-22 00:20 /tmp
drwxr-xr-x   - hdfs       supergroup          0 2015-10-20 21:04 /user

The python code fails
>>> import os
>>> from snakebite.client import Client
>>> c = Client('ip-10-0-0-166.us-west-1.compute.internal', use_sasl=True)
>>> os.system("klist")
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: edh@HADOOP

Valid starting       Expires              Service principal
10/22/2015 22:23:31  10/23/2015 22:23:31  krbtgt/HADOOP@HADOOP
	renew until 10/29/2015 22:23:31
>>> list(c.ls(['/']))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/snakebite/client.py", line 152, in ls
    recurse=recurse):
  File "/usr/local/lib/python2.7/dist-packages/snakebite/client.py", line 1198, in _find_items
    fileinfo = self._get_file_info(path)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/client.py", line 1326, in _get_file_info
    return self.service.getFileInfo(request)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/service.py", line 35, in <lambda>
    rpc = lambda request, service=self, method=method.name: service.call(service_stub_class.__dict__[method], request)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/service.py", line 41, in call
    return method(self.service, controller, request)
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/service_reflection.py", line 267, in <lambda>
    self._StubMethod(inst, method, rpc_controller, request, callback))
  File "/usr/local/lib/python2.7/dist-packages/google/protobuf/service_reflection.py", line 284, in _StubMethod
    method_descriptor.output_type._concrete_class, callback)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/channel.py", line 435, in CallMethod
    self.get_connection(self.host, self.port)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/channel.py", line 241, in get_connection
    sasl_connected = sasl.connect()
  File "/usr/local/lib/python2.7/dist-packages/snakebite/rpc_sasl.py", line 97, in connect
    res = self._recv_sasl_message()
  File "/usr/local/lib/python2.7/dist-packages/snakebite/rpc_sasl.py", line 78, in _recv_sasl_message
    sasl_response = self._trans.parse_response(bytestream, RpcSaslProto)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/channel.py", line 411, in parse_response
    self.handle_error(header)
  File "/usr/local/lib/python2.7/dist-packages/snakebite/channel.py", line 414, in handle_error
    raise RequestError("\n".join([header.exceptionClassName, header.errorMsg]))
snakebite.errors.RequestError: javax.security.sasl.SaslException
Client mechanism is malformed

Same result when running the code via python test.py (without the repl)
from snakebite.client import Client
c = Client('ip-10-0-0-166.us-west-1.compute.internal', use_sasl=True)
print list(c.ls(['/']))

@eschlon use AutoConfigClient. http://snakebite.readthedocs.org/en/latest/client.html#snakebite.client.AutoConfigClient

@praveev Thanks, I will try that tomorrow morning and report back.

 AutoConfigClient works like a charm.

Ill check whether this is really an issue with snakebite or a user config issue.

 AutoConfigClient works like a charm.
—
Reply to this email directly or view it on GitHub https://github.com/spotify/snakebite/issues/175#issuecomment-150641851.

Any update on this issue?

Any update on this? I ran into the same issue.
This doesn't work:
nn = [Namenode('dilithiumblue-nn1.blue.ygrid.yahoo.com', 8020, 9)]
client = HAClient(nn, use_trash=True, effective_user=None, use_sasl=True)

while this work:
client = AutoConfigClient()

I checked the config AutoConfigClient generated and it's exactly the same as the values I passed to HAClient.

@praveev @eschlon @bolkedebruin @wouterdebie
Did you guys have any update on this? Please see my post above.
AutoConfigClient worked for me and I printed out the config in AutoConfigClient. Then I used those values and created client with HAClient and Client but they both returned the javax.security.sasl.SaslException. Client mechanism is malformed error message.
It seemed really weird since AutoConfigClient is essentially calling HAClient with some params. If I can HAClient manually with those params, it just didn't work.
Really appreciate any help or pointers!
BTW I'm trying to get Airflow to work, but the code above is not related to Airflow so the issue should be Airflow-independent.

I think I found the cause of the problem (in my case).
__init__ in AutoConfigClient calls:
configs = HDFSConfig.get_external_config()

which has side effects. In HDFSConfig.get_external_config() which calls read_hdfs_config(), there are two lines:
https://github.com/spotify/snakebite/blob/master/snakebite/config.py#L90
https://github.com/spotify/snakebite/blob/master/snakebite/config.py#L94
This is the only way cls.hdfs_namenode_principal gets set. If you use HAClient or Client to create the connection, cls.hdfs_namenode_principal will never get set. Thus the failure.
Client and HAClient should probably take hdfs_namenode_principal as argument or their __init__ should read the value in some other way.
Thoguths? @bolkedebruin @wouterdebie

@ravwojdyla any thoughts on this?

@garthcn I completely agree with your assessment and thanks for tracking this down. I wouldn't call it side effects that the hdfs principal is set by the auto configuration. Maybe a proper fix is the append to the configs, like in line 85. But that is just from looking at this piece of code, I haven't thought about the implications.

@bolkedebruin
We can probably make line 12~15 here into instance variables. Then read_hdfs_config() and read_core_config() can return something like:
{
    namenodes: [],
    use_trash: True,
    use_sasl: True,
    hdfs_namenode_principal: <principal>
}

Then, change Client, HAClient to take in hdfs_namenode_principal; and AutoConfigClient will read the auto-configured hdfs_namenode_principal from the dict returned above.
We also need to change all places where HDFSConfig.use_trash, HDFSConfig.use_sasl and HDFSConfig.hdfs_namenode_principal are used (e.g. here)

@garthcn to me that seems to most elegant solution. For airflow I will create a patch that allows you to specify the principal or to use the autoconfig functionality

@bolkedebruin sounds good.
@ravwojdyla @wouterdebie any thoughts on the proposal above?
I'm working on a pull request.

I just updated the pull request and it should be ready for review

Looks good to me. One slight thing is that I was a bit confused by the term configs (I know the term was used already). Maybe config_parameters is a better term? I
Still waiting on @ravwojdyla..

#192 was merged - once new version is released, and the issue is gone we could close the issue.

Thanks and also thanks to @garthcn

Thanks everyone!

I was having the same problem as in the OP and for me the actual problem turned out to be missing the cyrus-sasl-md5 and(/or?) cyrus-sasl-gssapi packages. Installing these packages solved it.

Can someone update the docs on this?


[Question] CCP for device settings

You want to change the device in the softphone settings. You want to set the ringer to the speaker and the customer's voice to the headset.
I'm developing a custom ccp, and the ccp doesn't give me an error when I get the device id,
In the custom ccp, I get an error because the device IDs don't match.
Error message in developer mode
"RingtoneEngineBase.setOutputDevice failed: audio.setSinkId() failed with error NotFoundError: Requested device not found"
I think the deviceId is in error, if you have any guide that can help, please share.
(Additionally) When I compare, I see that the device ID from which I get the device list from custom ccp is different from the device ID I get from stream.js.
Is there any function to get the deviceList using stream.js ?

When I compare, I see that the device ID from which I get the device list from custom ccp is different from the device ID I get from stream.js.
Is there any function to get the deviceList using stream.js ?

In your custom CCP, before you set the set your audio devices, you should validate that the device you intend to set is indeed available to use. One way you can do that is by using navigator.mediaDevices.enumerateDevices(). This will return all devices available for use. You can check if the device you want to set is in that list and if it is, then you can set it.

Hi @kimhanmiru , have you tried connect.core.getFrameMediaDevices() API?
The deviceId is expected to be unique per document/frame (see MDN), so you must capture the device list from the context that handles the softphone. If you allow CCP iframe to handle softphone, use the connect.core.getFrameMediaDevices() API. If you use RTCJS and handles softphone on your website context, use navigator.mediaDevices.enumerateDevices().


auto save doesn't work for typescript

Do you want to request a feature or report a bug?
bug
What is the current/expected behavior?
auto format should work for typescript
What version of vim-prettier are you using - (output of :PrettierVersion) ?
1.0.0-beta
What version of prettier are you using - (output of  :PrettierCliVersion) ?
2.1.2
What is your prettier executable path - (output of  :PrettierCliPath) ?
/Users/myname/workspace/myproject/node_modules/.bin/prettier
Did this work in previous versions of vim-prettier and/or prettier ?
it's my first time with vim-prettier
:Prettier work fine.
autosave works for the other langs(js, json ...) but no for the typescript
here's my config.
let g:prettier#autoformat = 1
let g:prettier#autoformat_config_present = 0
let g:prettier#autoformat_require_pragma = 0
let g:prettier#autoformat_config_files = ['javascript', 'typescript', 'javascriptreact', 'typescriptreact', 'css', 'less', 'scss', 'json', 'graphql', 'markdown', 'vue', 'yaml', 'html']

closing. it was vim-styled-component breaking the typescript.
after removing that, vim-prettier works great.
good!


Use a single admin boundary file for styling

The PR #284 inadvertently added a lot of complexity to how we manage admin boundaries. It became apparent with #338
Instead, we should handle this behavior in the app. For example at build time, using turf.dissolve or turf.union and next getStaticProps.
Here is a working example (to improve, using union)
  // Group features and merge them by admin level. Here "ST" for testing.
  const firstLevelData = data.features.reduce(
    (acc: Record<string, GeoJsonBoundary>, f) => {
      const feature = f as GeoJsonBoundary;
      const levelKey: string = get(feature, ['properties', 'ST'], 'no_level');
      const existingData = get(acc, [levelKey]);
      // console.log({ levelKey, existingData });
      try {
        const newAcc = {
          ...acc,
          [levelKey]: existingData
            ? (union(existingData, feature) as GeoJsonBoundary)
            : feature,
        };
        return newAcc;
      } catch {
        return acc;
      }
    },
    {} as Record<string, GeoJsonBoundary>,
  );

Using dissolve should be faster but requires converting all multipolygons to polygons first.
Some other tools to explore such Mapbox Expression Formatting for line-width:
[
    "case",
    ["has", "TS"],
    0.5,
    ["has", "DT"],
    1,
    ["has", "ST"],
    1.5,
    // default
    0.5
]

We could also adapt https://github.com/Turfjs/turf/blob/master/packages/turf-concave/lib/turf-polygon-dissolve.ts#L17 which uses topojson as an intermediate state

I opened https://github.com/WFP-VAM/prism-frontend/tree/admin-styling-tests to investigate a few options.

It might be worth connecting with @JorgeMartinezG as he's working on issue https://github.com/WFP-VAM/prism-app/issues/587 and I think might be revisiting how we work with admin levels


Can I login or signup with attribute other than email like username?

I want to be able to login using a username or an email whatever the user chooses. I want conditions that the user I pass a single field like login in the request API parameters and the user model returns first user that matches the values either in username or email.

you may right, you contribute to this gem for this functionality @Sovaid-Shah :)

As mentioned in #40, i have submitted a PR to fix this request.

#40 merged, you can use now @Sovaid-Shah 🙌🏻


It seems that there is no use for adversarial losses

Thank you for your code. I read the code, but can't find the adversarial losses anywhere. The _NetD network is not used, right?

@Alxemade https://github.com/twtygqyy/pytorch-SRResNet/blob/srgan/main_srgan.py#L211


Body helper enhancement - Stringify

Adds a new parameter to the body helper.
If it is set to true and the value is a primitive, the helper returns the stringified version of the value.
This fixes an issue where string values where not correctly escaped when being returned by a body helper. The reason for going with an extra parameter here is to 1. ensure backwards compatibility and 2. still enable user to use the unescaped values as the input to other helpers like concat.
Parent issue
Closes https://github.com/mockoon/mockoon/issues/397
(needs to be closed manually)
Technical implementation details
There is a third parameter for the body helper now. It is false by default and you can set the value by invoking it via {{ body 'foo.bar' undefined true }}. If the parameter is set to true, the value referenced by the path is processed by JSON.stringifybefore it is returned. If parameter is false, primitive values (i.e. null, integer, string, boolean) will not be processed by JSON.stringify. However, objects and arrays will always be stringified, no matter which value the user passes for the parameter.

@255kb One thing that might need to be discussed is whether or not the default value should also be stringified if the parameter is set to true.

@255kb Good catch! I fixed the issue and added a test case. I also incorporated the change that the default value gets stringified.

Thanks for the changes @mauriceackel, it looks great!
One heads up concerning this repo: I am currently working on some big architecture refactoring linked to how Electron expect things to work. The result is that I splitted mockoon/commons in two, creating a new mockoon/commons-server with the helpers code. So I will move everything there in the coming days.


fix default encoding for bytes

We want to default to no encoding for a bytes input body, but also default to base64 encoding for a bytes property on a json object

Change to use content-type to judge if bytes array body need to be serialized.

Pending on cadl-ranch release: https://github.com/Azure/cadl-ranch/pull/424

part of: https://github.com/Azure/autorest.python/issues/2038


Moved GitHub reference lower in the list of Support Types

Hopefully, encouraging developers to use other resources before GitHub

@EricDahlvang,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

@EricDahlvang can you resolve the merge conflict? thx


htslib: 1.11 -> 1.12

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/htslib/versions.
meta.description for htslib is: "A C library for reading/writing high-throughput sequencing data"
meta.homepage for htslib is: "http://www.htslib.org/"
meta.changelog for htslib is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.12 with grep in /nix/store/sn6zm7bq69m8kxmdfy3kw248v694fv9d-htslib-1.12

directory tree listing: https://gist.github.com/035027c65e6bd82e1cf5f776333f4a3c

du listing: https://gist.github.com/2bf1ed5b14e4832047a659190639e03c

Rebuild report (if merged into master) (click to expand)

53 total rebuild path(s)

17 package rebuild(s)

17 x86_64-linux rebuild(s)
13 i686-linux rebuild(s)
12 x86_64-darwin rebuild(s)
11 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
bcftools
deeptools
delly
htslib
lumpy
mosdepth
octopus-caller
platypus
python38Packages.HTSeq
python38Packages.cnvkit
python38Packages.pysam
python39Packages.HTSeq
python39Packages.cnvkit
python39Packages.pysam
samtools
tebreak
truvari

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/sn6zm7bq69m8kxmdfy3kw248v694fv9d-htslib-1.12 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A htslib https://github.com/r-ryantm/nixpkgs/archive/d7521a0324c41b3ee17712dc3146789cbe891857.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/sn6zm7bq69m8kxmdfy3kw248v694fv9d-htslib-1.12
ls -la /nix/store/sn6zm7bq69m8kxmdfy3kw248v694fv9d-htslib-1.12/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

11 packages failed to build:

deeptools
lumpy
python38Packages.HTSeq
python38Packages.cnvkit
python38Packages.pysam
python39Packages.HTSeq
python39Packages.cnvkit
python39Packages.pysam
samtools
tebreak
truvari

6 packages built:

bcftools
delly
htslib
mosdepth
octopus-caller
platypus

Maintainer pings
cc @mimame for testing.

Result of nixpkgs-review pr 116704 at d7521a03 run on aarch64-linux 1

6 packages marked as broken and skipped:

deeptools
python38Packages.HTSeq
python38Packages.cnvkit
python39Packages.HTSeq
python39Packages.cnvkit
truvari

1 package failed to build (new failures):

samtools

4 packages built successfully:

bcftools
delly
htslib
mosdepth

Result of nixpkgs-review pr 116704 run on x86_64-linux 1

11 packages failed to build:

deeptools
lumpy
python38Packages.HTSeq
python38Packages.cnvkit
python38Packages.pysam
python39Packages.HTSeq
python39Packages.cnvkit
python39Packages.pysam
samtools
tebreak
truvari

6 packages built:

bcftools
delly
htslib
mosdepth
octopus-caller
platypus

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 116704 run on x86_64-linux 1

11 packages failed to build and are new build failures:

deeptools: log was empty
lumpy: log was empty
python38Packages.HTSeq: log was empty
python38Packages.cnvkit: log was empty
python38Packages.pysam: log was empty
python39Packages.HTSeq: log was empty
python39Packages.cnvkit: log was empty
python39Packages.pysam: log was empty
samtools: log was empty
tebreak: log was empty
truvari: log was empty

6 packages built:

bcftools
delly
htslib
mosdepth
octopus-caller
platypus

Build has a lot of failures, but seems as if it's not necessarily the fault of this package or this version bump. Can we get a review by someone who is familiar with the package and use-case?

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 116704 run on x86_64-linux 1

2 packages failed to build and already failed to build on hydra master:

python38Packages.cnvkit: log was empty
python39Packages.cnvkit: log was empty

9 packages failed to build and are new build failures:

deeptools: log was empty
lumpy: log was empty
python38Packages.HTSeq: log was empty
python38Packages.pysam: log was empty
python39Packages.HTSeq: log was empty
python39Packages.pysam: log was empty
samtools: log was empty
tebreak: log was empty
truvari: log was empty

6 packages built:

bcftools
delly
htslib
mosdepth
octopus-caller
platypus

new release available


Lane Detection

Different approaches possible:

Extract polynomial using different line detection algorithms or Machine Learning
Road segmentation
(add more if you have an idea)

Maybe that can be helpful:
https://github.com/tatsuyah/Lane-Lines-Detection-Python-OpenCV


Create chat screen UI using Material UI(https://material-ui.com/)

Branch: Development1.0
File: components/Layout/ChatBox.js
Current Behavior: On alumni page clicking on any chat icon opens a fullscreen dialog without any actual content.
Expected Behaviour: Design a chat UI similar to the this. Improvizations are welcome :)

I'll work on this

sure @PabloPap , make sure you work on Development branch and create a pull request to Development branch only.

hi @PabloPap , r u still working on this?

hi @honeycoder96, due to shortage of time I can't keep up for now....You can assign someone else

OK problem @PabloPap thank anyway.

hey @honeycoder96 ! Just started with this repo to check how hard is this feature to implement!
I'm wondering what is .history dir?

I am interested in working on this issue. It doesn't look like there are any api routes set up for storing any of the chat dialogue? Can I just submit a pull request with dummy data atm?


Support workspace clone across Terraform Organizations

It returns a 404 error when I select a different organisation for the target workspace.

The problem was with Terraform Organization API token! It works with user API token. Closing the issue!


fix interface conversion panic

Methods currently throw interface conversion panics. See MergedState() return types.
panic: interface conversion: interface is []api.Endpoints, not map[string]map[string]api.Endpoints

goroutine 1 [running]:
github.com/GoogleCloudPlatform/kubernetes/pkg/proxy/config.(*EndpointsConfig).Config(0xc20800aee0, 0x0)

LGTM


Round values to 0.5

Is there function to round values to a given precisions?
I want all my values(temperatures) to be rounded to 0.5
Thank you!

Today I was missing this function as well. It would be nice to have a rounding function for timeseries.

Would help to have round and trim functions..

Is this implemented now? Why has this been closed?

No, it was not implemented. Not sure why @eliabieri closed it.
Newer Graphite versions have support for custom functions - see docs.
You can also open PR and we'll include function in next release (but in that case it should also have tests, documentation etc)

@monsdar : you can also check my example for custom function - https://github.com/deniszh/graphite_asap

This could be harder to implement correctly than it looks as Graphite uses floats internally. I don't know if this will be an issue in practice but it could lead to unexpected results.


Broken for Home Assistant 2021.6

This integration does not work with Home Assistant 2021.6.
The following error is showing up in the log:
[custom_components.citymind_water_meter.managers.entity_manager] Failed to update, step: Start updating domain sensor, Error: 'sensor', Line: 167

Works for me, can you please post logs (with debug level)
Thanks

@maorcc, checked it once again, I'm running v1.0.0,
so most probably it is related to v1.0.1 changes

Same issue here.
Logger: custom_components.citymind_water_meter.managers.entity_manager
Source: custom_components/citymind_water_meter/managers/entity_manager.py:424
Failed to update, step: Start updating domain sensor, Error: 'sensor', Line: 167
V1.0.1

@elad-bar, did you upgrade Home Assistant to 2021.6?
Before I created the Citimind v1.0.1 release, it didn't load on 2021.6 because there was no version attribute in the manifest file.  That's why I created the v1.0.1 release.
I am not sure what's the cause of the new error.  Maybe related to the minor changes that were done recently.  @yuvalabou, can you have a look too?

Yes I did, HA 2021.6

@maorcc, i'm on 2021.6 + 1.0.0 (with local version key change) and it works, most probably changes of 1.0.1 led to it, will try to check it during the weekend

I have the same error when running 1.0.1 on HA 2021.5.5

I have the same issue with 1.0.1 and 2021.6
It seems broken on 2021.6

Just fixed it by rolling back to version 1.0.0, and setting the version manually in the manifest.json file.

https://github.com/maorcc/citymind_water_meter/pull/18

Fixed in v1.0.2 by @yuvalabou, please update if it works,
thanks

Working as expected


Allow UI cycling through option sets

There are multiple places where different analyses or presentations could be configured.  Having one prescription for all dataframes, and requiring BuckarooWidget to be reinstantiated/configured for each type is cumbersome.
This is particularly relevant for

Auto-cleaning (off, aggressive, types only...)
Summary stats different sets of summary stats
Sampling methodology (random, first 500, segments of 50 each)
histogram presentation styles
column ordering
sets of low code UI commands

Auto-cleaning is probably the best candidate.
The function should work such that the backend suppplies a list of options and current, then clicking on that value toggles through the different options, changing the bi-directional widget value of current.

This is possible. look at https://github.com/paddymul/buckaroo/blob/main/example-notebooks/Extending.ipynb
Look at the kitchen sink example at the bottom
Fixed by https://github.com/paddymul/buckaroo/pull/201 for release 0.6.1


Add example 'contact us' package

I was getting weird babel runtime errors on a new install and a version notch fixed it. It probably has something to do with Meteor. Here is a link to a Meteor forum post addressing it
Add a package that utilizes the VulcanEmail .send function to send a confirmation email when a guest wants contact with the 'site admins'. Useful for websites where the customer wants to be in contact with the 'site admins' (e.g. the HR or support team)
The mutation check on collection.js simply returns the document. This might be too unsafe.

Thanks, I'll check it out :)

I've updated it with a description. It should be quite easy to get started now :smile:

Sorry, finally had time to look at this!
I like the email integration but I wonder if it wouldn't be better to rethink the example a bit. First of all, I think the naming has some issues, as the collection is called ContactUsForm. It should probably be called something like Messages instead.
But beyond that, I think it'd be more valuable to not use a collection or SmartForm, and instead use GraphQL mutations to submit the form (even if that might not be how you'd do it in a real world app). This would both match the mental model of how a normal contact form works better; and also showcase something that isn't available in other existing examples.
I think we can also drop the rate limiting from the example, as that's something that's not really related to Vulcan directly, and I'd rather leave it up to people to find their own implementation.
I like having the custom validation and custom error in there though, we could keep this but just hook it up to the flash messages instead of SmartForm.
Oh and as a final note it'd be better to use the email templates system rather than just pass raw HTML to VulcanEmail.send. After all the goal of the examples is to showcase Vulcan best practices.
Hope it makes sense, please let me know if you have any questions!

Would ContactUsMessages be fine? The point of this package is to use the code as-is and simply plug it into an existing app.
The rate-limiting file can be deleted if some other functionality is needed/used, why shouldn't it be there by default? It's just bonus boilerplate code, it simply makes it easier to use immediately in production.
I tried getting help for the template system but it's really hard to make something. Also, the way these emails are structured is basically the same, just in native template strings. If the documentation actually guided towards making your own functionality and I can figure out how to use templates without spending several hours reading through source code - the original functionality can be used. furthermore, these emails are just raw HTML. They're not supposed to be filled with all kinds of fluff, it's a nice extra for the customer to receive an email and a more effective way for the admin to answer quickly without problems. But, as said, I'll do it if I don't have to spend several hours looking through the source code to troubleshoot problems.
I hope that you are going to reason your arguments. I might come off as rude but you're basically just saying words based on your opinion. There's no reasoning, your message is that I've done most in
a different way you are thinking and therefore it is wrong and should be changed.

Regarding rate-limiting - it's needed no matter what. Anyone is able to send a mail and if there's not rate-limiting from the beginning I have the moral responsibility when thousands of spam-bots use this exact functionality and it could have been prevented by have rate-limiting on by default - making sure people opt out instead of not thinking about it at all or when it's too late.

Well, the role of these examples is to teach people about the various features of Vulcan, that's why it's important to use them. Regarding the email thing, if people see plain HTML text strings they might deduce that it's the "correct" way of doing things in Vulcan even though that's not really true.
That's also why it's not really necessary to make this a drop-in package. The goal is not for people to use as is, but to understand the code and adapt it to their own needs (of course you can also make a separate drop-in plugin and publish it as a Meteor package, that would be great too!).
For the rate limiting I guess if it was me I wouldn't include it, but it's not a big deal either way. If you feel it's better with it I don't mind.


Features/first release limitations

Some documentation about the first release limitations

@fitzthum Does this capture our discussion today?
@ariel-adam I added some terminology or rewording for the terms that I thought would be unknown to newcomers.


Feature Request: shpool attach with no args? Pick the most recent un-attached session

The idea for this convenience feature is taken from tmux: it'd be nice to be able to simply shpool attach (or better yet, shpool a) when there's 1+ unattached sessions and have shpool just pick one to attach to automatically.

This seems like a reasonable feature. We could do this by adding a last_connected_at timestamp to the session table and then issuing a list rpc to choose the most recently accessed session.


WIP - V2

Still to fix for this PR...

[ ] Spellcheck
[ ] Registries
[ ] Generator
[ ] Proper testing

@nadilas take a look here, I've done some work on simplifying Lile's service stuff and registry interface so it should be easier for you to implement consul and zoopker now.
Let me know what you think about the changes? Happy to help out


:app:installdebug error

Hi i followed all the steps

i was waiting more than 25 min also no result
and also the apk file size is more than 500 MB

also my code is easy and understood!

any one can help
im using react-native last version
and i got this screen in my phone

and this is my image

I updated the issue with the final result plz help me i need ocr for my end of studies project :'(


Fixed #33640

Signed-off-by: Himanshu Balasamanta<EMAIL_ADDRESS>

Closing per ticket.


Feature/inherit config support

Support a config file generated dynamically.
In a basic usage, we have all config files to be registered to luigi initially when starting docker container from an image, since we usually copy these files to container local.
However, in some advanced cases, we generate additional config files dynamically after starting docker container.
This kind of file is not specified in Dockerfile, so when starting child jobs, they do not have this file and failed when utilizing the dynamically generated config files.
So, this PR adds a feature where the master job passes the config info to children via gokart cache.
Currently, only one file is supported, but in the future we'll support multiple dynamic config files / other format of files.

We replace TASK_WORKSPACE_DIRECTORY with root_task.workspace_directory constant.
Requiring TASK_WORKSPACE_DIRECTORY is not so good since the kannon library itself limit the name and usage of the env var. This env var has been used to get the path of remote cache like GCS and S3. It can be replaced with root_task.workspace_directory for the same purpose.
Note that users are still required to specify remote workspace directory info via ini config files. When you use some env vars in them, you have to pass them to env_to_inherit.