[Discussion] class-methods-use-this

https://github.com/Shopify/javascript#12.13

12.13  Instance methods that don’t refer to this don’t need to be instance methods. If they relate to the class, make them static methods; otherwise, make them functions in scope.

Why? This pattern is generally a sign that you are providing a bad public API for the class, and should either hide this method (if it’s an implementation detail) or export it as a utility method.

ESLint rule: class-methods-use-this
// bad
class BadClass {
  badMethod(string) {
    console.log(string.toUpperCase());
  }

  anotherMethod() {
    return this.badMethod('oops!');
  }
}

// good
function goodFunction(string) {
  console.log(string.toUpperCase());
}

class GoodClass {
  anotherMethod() {
    return goodFunction('oops!');
  }
}

This rule bothers me a little. While I understand the intent, moving instance methods to named functions outside of the class

Doesn't automatically make them reusable (their functionality could be specific to what the instance does

Makes code harder to read as you can no longer group related functions together

Makes classes harder to subclass (as you have no way of reusing parts of the internal implementation) - this is probably a good thing though

Forces you to export those methods if you want to test them directly (which makes the module surface area larger than it needs to be)

Why? This pattern is generally a sign that you are providing a bad public API for the class, and should either hide this method (if it’s an implementation detail) or export it as a utility method.

While I agree this would be useful in the case of libraries/SDKs, we already signal internal vs external methods using the leading underscore convention

2.3  Use a leading underscore when naming "private" properties. Functions and variables in scope should be named normally.

Why? The leading underscore sends a signal to other developers that these methods should not be called or relied upon. Some tools can also obfuscate methods with leading underscores to ensure that they are not called by outside objects. Items in scope but not exported are completely private, so no signaling is required for these.

Note: Use these underscore-prefixed members as a last resort. Prefer moving them to be functions/ variables in scope or making them part of the public API over using this naming convention.

Also, TypeScript should help with this in the long run.

For instance, in the following class
class ListComponent {
  constructor(items) {
    this.items = items;
  }

  render() {
    const list = document.createElement('ul');
    this.items.forEach((item) => list.appendChild(this._renderItem(item)));
    return list;
  }

  _renderItem({body, id}) {
    const item = document.createElement('li');
    item.innerText = body;
    item.dataset.id = id;
    return item;
  }
}

_renderItem() would caught by the class-methods-use-this linting rule, forcing the author to refactor to something like
function renderItem({body, id}) {
  const item = document.createElement('li');
  item.innerText = body;
  item.dataset.id = id;
  return item;
}

class ListComponent {
  constructor(items) {
    this.items = items;
  }

  render() {
    const list = document.createElement('ul');
    this.items.forEach((item) => list.appendChild(renderItem(item)));
    return list;
  }
}

While we now have the advantage of having renderItem being completely private, it's also completely outside of it's usage context and we leave it up to the reader to jump between both and figure out how it's being called.

Great idea creating an issue for this topic, this has come up once or twice before and would be nice to have a canonical issue to reference when this decision becomes questioned.
My take on this:
1.

Doesn't automatically make them reusable (their functionality could be specific to what the instance does

Increased reusability isn't the intent of this rule. I believe it is more to make you question whether your method is actually dependant on the instance (using this) or not. If not, the behaviour of the function likely shouldn't be available as a method on the instance.
2.

Makes code harder to read as you can no longer group related functions together

I think this the core of the reason for this lint rule and would like to challenge your assumption here. It was an improper grouping to begin with, the fact that the function does not contain any reference to this is the smell that the function isn't related to the instance and should be pulled off the class.
There is however nothing to stop you from grouping related helper functions outside the class
const AwesomeHelpers = {
  doThing: options => { /* ... */ },
  doOtherThing: (arg1, arg2) => { /* ... */ },
}

export default class AwesomeClass {
  // ...
}

3.

Makes classes harder to subclass (as you have no way of reusing parts of the internal implementation) - this is probably a good thing though

I agree, this is a good thing. If the subclass needs that helper function you can always export it from the original file as well as the class, or include it in a separate file containing related helpers.
This is JS though, so we should always consider other reuse patterns like mixins or composition. I've always preferred the philosophy that inheritance is more of a pattern for deviation/special cases, vs reusability.
4.

Forces you to export those methods if you want to test them directly (which makes the module surface area larger than it needs to be)

I would disagree here. The private, not exported function does not need to be tested. We should only be testing the public interface against our use cases. As @m-ux has said before: "test the interface, not the implementation"
5.

Why? This pattern is generally a sign that you are providing a bad public API for the class, and should either hide this method (if it’s an implementation detail) or export it as a utility method.

While I agree this would be useful in the case of libraries/SDKs, we already signal internal vs external methods using the leading underscore convention

The leading underscore for private methods is really more of a shitty workaround for not actually having private functions available in ES5. Given that we can now hide them inside modules using closure, it's less of a point and we should probably stop using this leading underscore pattern.
If a function is hidden/not visible there is no longer a need to differentiate it to the consumer of your API.

My main takeaway from this rule is that it challenges your assumptions about a specific instance method and tells you that you're wrong without emotion or tact. You should embrace this robotic calousness as a chance to rethink how and why you wrote that function in the way that you did.
Overall I think we can and should be shipping the minimal functional interface rather than adding confusing public methods that are privately named and hoping that no one implements them.

A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.
-- Antoine de Saint-Exupery

@perrupa did a good job of voicing my usual points in favour of keeping this rule. In general, I have seen this rule be a problem in the following cases:

Someone trying to write a JS class as they would a Ruby class, where you are expected to attach all of your functions to some sort of class. This is not necessary in JS because we have a different method of organization (modules), and because "floating" functions are a more expected part of JS.
Someone using a base class as an interface, where they expect subclasses to override empty/ error-throwing versions of the methods in the superclass. This is definitely a code smell in JS, I would question someone doing even one additional level of class hierarchy in almost every case
Needing a method to stub out interaction with some global in test (for example, a method that sets window.location, which is the most common one I have seen). This is not strictly necessary in most case (can stub out global functions, though not always consistently in every environment), but in some cases it is unavoidable

Of these three, I think that 3 is acceptable, but the other two are legitimately bad patterns that this rule actively discourages. I like rules that generally encourage good code patterns, so my preference would be to keep this rule.
(as an aside, TypeScript does have private members, but we have still been moving functions that don't reference this off the class. It cleans up the class, and methods that don't need this are often either incomplete, rely on a global that will make the component hard to test, and/ or are a piece of functionality that many components could use, so it makes sense as a utility)

There's a solution to this 😛
https://github.com/Shopify/shopify/search?utf8=✓&q=class-methods-use-this

Consistency and readability is paramount and this rule makes code far less readable by requiring some private methods (but not all) to be broken out into floating functions.
Having floating functions as generic (globally or locally scoped) helpers makes sense. Having floating functions performing work that’s specific to a class does not. It’s the quintessential example of blindly adhering to a rule for the sake of that rule.
Some real effects of this rule on my code, which has been evolving as requirements change, have been:

Having to convert and move private methods because they no longer refer to this, affecting the coherence and readability of the module.
Having to remove unit tests that were testing the now converted method, reducing the coverage of the module and making it more brittle.
Having to convert functions back into private methods because they’re now referring to this again.
Having to add back tests that were previously removed.

And as far as unit testing private methods goes, when 75% of my module’s functionality are private methods, I’m going to unit test them to make sure things don’t break as my module evolves. I need to protect myself against myself over time.

I feel all of your points to not go to addressing my central concern about removing this rule. I would ask you the following for the effects this has on your code:

Why do your methods no longer refer to this? Are you sure you actually need a class if a significant number of methods do not use this?
Why do you want to test private members? This is considered a smell in almost every language, it just happens to be easy to do in JS. Why aren't you just testing your public API? Are the methods you are breaking out really just utilities that should be separately tested as pure functions?
When does this happen? Are you sure your class is splitting up code in an appropriate way (it's usually better to isolate stateful code to a pretty small number of places, not have it in every method).
Not sure I totally understand this one.

As I noted in my original response, I think there are some cases where this can make sense, but I feel they are generally exceptions, and that the common case here is helping to identify smelly code.

Honestly, this rule causes enough headache that I'd be fine taking it off. The thing I don't want to lose is that people continue to ask the questions I asked above when they see methods that don't use this — it's not always a code smell, but often is. If the team has strong agreement on this being the right way forward, please make the PR and add some content to the style guide that makes it clear what situations a method without this makes sense in, to generally limit how many methods mutating this, and what situations you should a) not use a class or b) use utility functions instead.

Did some digging to figure out how to update the rule.

We have it configured as a warn which shouldn't affect the exit code,
But then we run ESLint with --max-warnings=0

I would have expected warn rules to be used to suggest that the offending code might be a code smell rather than enforcing the rule all the time (which is exactly what we want, based on the above discussion). They should show up in your editor but not prevent you from merging.
@lemonmade should I remove the rule altogether or should we update the rules to correctly reflect what should be a warn and what should be an error?

For me, the rule flags meaningful information.  As @lemonmade detailed, there are valid alternative approaches to consider.
If you've validated your approach, or you're just quickly iterating in a branch, the rule can be disabled, no?

@davidcornu we have a weird split between error/ warning that comes out of Atom originally doing horrible things if you just have everything as errors. The basic rule of thumb is that everything we turn on should fail CI, warnings should be for things that might logically come up during the development flow that are not likely to be logical errors (so, most stylistic rules). If we are turning it off, we should turn it all the way off.

I don't like this rule because it often encourages developers to use worse patterns than those that it claims to discourage.
I've seen people get around these warnings by putting all the logic that doesn't have 'this' into a separate utility object; this makes it hard to figure out which object to use when you want to perform certain related operations.
It's much easier to make sense of the code if closely related functions are grouped under a single object instead of two seemingly unrelated objects.
The internal implementation details of the method should not be a concern to the user of the method.
The priority should be to make sure that exposing similar functionality is done in a consistent way. Just because a method doesn't use 'this', it doesn't necessarily mean that it should be a static method (that's a separate concern and requires critical thinking).
For example, if I have two related methods like startNode() and stopNode() and one happens to use 'this' but not the other, then doesn't mean that one should be myInstance.startNode() and the other should be MyClass.stopNode().
This rule is overfitted; it probably good in some cases but it's bad in others. It's not obvious at all what it expects from us and so people get different ideas... Sometimes really bad ideas.

Was resolved in
https://github.com/Shopify/eslint-plugin-shopify/pull/45
https://github.com/Shopify/javascript/pull/266
Closing


Excel files with inlineStr cell types read empty value

[Description]
Excel files with inlineStr cell types read empty value, here is the sample file
output.xlsx
Steps to reproduce the issue:
1.
download the sample file and read it with sample code as belong
2.
xlsx, err := excelize.OpenFile("./output.xlsx")
if err != nil {
fmt.Println(err)
return
}
rows := xlsx.GetRows("test")
for _, row := range rows {
for _, colCell := range row {
fmt.Print(colCell, "\t")
}
fmt.Println()
}
Describe the results you received:
1	2	3
4	5	6
7	8	9
Describe the results you expected:
col1	col2	col3
1	2	3
4	5	6
7	8	9
Output of go version:
any
Excelize version or commit ID:
Excelize version: 1.2
Environment details (OS, Microsoft Excel™ version, physical, etc.):
any

Thanks for your issue, I have added read inlineStr type cell value support. I also saw your pull request after I fixed it, thanks for you do that. I will close the PR.

Thanks for your support, really fast and terrific.


add NextPart benchmark

Added benchmark when using NextPart of standard library.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 72.16%. Comparing base (0b5ca3b) to head (66564e9).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #17      +/-   ##
==========================================
+ Coverage   70.55%   72.16%   +1.61%
==========================================
  Files           8        8
  Lines         309      309
==========================================
+ Hits          218      223       +5
+ Misses         69       65       -4
+ Partials       22       21       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


improved readme commands

would be easier to actually ssh into containers using docker-compose. using
docker-compose exec nginx bash

for instance.

@AminMkh still haven't merged this because it didn't worked on my computer, I though you probably had an updated version of docker compose. I was waiting to update mine, test it, confirm it's working and merge. Sorry for taking too long since I'm very busy these days working on some projects.. Thanks for your contributions :)

all good, it is a new feature, and even the docker website's documentation isn't updated yet.
I can only see it here
but not here nor here
docker-compose -v
docker-compose version 1.7.1, build 0a9ab35

docker-compose
Commands:
  build              Build or rebuild services
  config             Validate and view the compose file
  create             Create services
  down               Stop and remove containers, networks, images, and volumes
  events             Receive real time events from containers
  exec               Execute a command in a running container
  help               Get help on a command
  kill               Kill containers
  logs               View output from containers
  pause              Pause services
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pulls service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  unpause            Unpause services
  up                 Create and start containers
  version            Show the Docker-Compose version information

Hi @AminMkh, I will close this PR in favor of #115 as it worked just fine!

These changes are now implemented in LaraDock 4.
Many thanks.


Performance regression in mj_copyData (2.3.0 vs 2.1.0 )

Hi,
I recently migrated a mujoco project from mujoco version 2.1.0 to 2.3.0. I discovered that my application is running way slower. I ran a performance profiler to see why.

Turns out mj_copyData is way slower than before.
For those who are wondering about my motivations and why I'm doing this: I am copying the data, because i have 2 threads one in which i run the simulation and the main thread where i render/visualize.
Reproduce: I can provide a minimal working example if needed. However in this case I think it should be very easy to reproduce and compare. But if it isn't, I can provide something just let me know.

Include the following context:

Windows 10 x64
MuJoCo version 2.3.0 vs MuJoCo version 2.1.0

Thanks, we will look into this.
In order to help us diagnose, could you please print mjData.nstack and mjData.nbuffer in both cases?
However, you should know that mj_copyData is not the recommended way to duplicate an mjData. This is mentioned here. As explained here, MuJoCo has a well-defined state. You shuld just be copying over the state. In your case, for visualisation purposes, it is enough to copy qpos and then mj_fwdPosition, or if you want to visualise forces then copy the entire integration state and call mj_forward. I expect this will be faster than mj_copyData. If you still want to use mj_copyData, perhaps for convenience, you can make it faster by reducing allocated memory. The latest version of simulate (not in any release, at head) reports the maximum amount of memory actually used. You reduce memory by using the <size memory/> directive. Look at 5004a4c8c5468891b5a8254cb867b70ebf683cbb for examples.

Hi Yuval,
thanks for looking into this and also thx for pointing out the more efficent way.
Diagnostic information:
mujoco version 2.1.0:
nbuffer: 5052592
nstack: 2062480
mujoco version 2.3.0:
nbuffer: 285568
nstack: 2663290

@ThoenigAdrian I find it rather suspicious that mj_step is 3x faster now compared to 2.1.0. I'm not aware of anything that we've done that could've caused it to run quite that much faster.
Are your numbers reliably reproducible? Would you mind running it against a few different versions in between?

Yes it's reliable reproducible.
I can understand your suspicion. I thought the same in the beginning.
But this is just because mj_copyData takes longer therefore mj_step has less time to run.
And since the time I let my application is fixed (around 20 seconds) less time is spent in mj_step

I currently don't have enough time to run it against a few different versions. But when I do in the future I'll let you know the results.

@yuvaltassa copying the state and recreating the contents of mjData works fine for me.


Removing decimals

Wish there could be a way to disable decimals so all transactions in-game are with hard digits. I'm not sure how to explain this for you to understand it better but the core concept here is -- all integers, no doubles/floats/whatever. I don't want people to buy/sell stuff with a price of $1.5 for example, it's either 1 or 2 (integers), nothing in between.

That would be pretty cool. Or in general that we can determine the amount of decimals.

I have the opposite situation. Maybe you should added an option.

Yeah an option is always good.

That is the plan, I hope to setup a formatting option so you can complete customize how currency is displayed.

Any updates on this?


Added user-defined context variable guard

Fitting with the "don't pay for what you don't use" concept, I wanted to have an optional way of injecting some kind of lock/guard around context variables in the registry.
I let the user define how the lock is created since, for me, I use a fiber mutex rather than std::mutex, so I need to be able to define it myself. I'm using the lock in the 'leaf nodes' of the function calls since I can't support recursive mutexes and the end result is the same.
Addresses #799
(Let me know if you want a PR against another branch; maybe main?)

Unfortunately, this PR is no longer valid (if we consider branch wip as it ought to be, that is about to be merged with master).
Briefly, I've introduced a registry_context object here to address #575 (that makes perfectly sense to me). All functions from the registry have been moved to this object. These are the only functions available in the registry now.
I've no problem merging or reproducing the intent of this PR on a branch, adapting it and then porting it to master. There is no need for you to intervene. However, I wanted to ask you what your plans are to avoid conflicts. 🙂

Actually, another option could be to turn the context type in a template parameter.
This way users can inject their own guarded maps or whatever or even suppress it. 🤔

@skypjack I actually had to comment out the
#ifndef ENTT_REGISTRY_CONTEXT_GUARD 	#define ENTT_REGISTRY_CONTEXT_GUARD() #endif
section as include ordering was messing with it - some files/libraries would get to config via various utilities and I was struggling to always keep my own header/define before it. I think there's probably a better solution on the cards, but I'd definitely like some way to inject a custom lock either way. A template param could be nice - means the user can typedef entt::registry and use it in their code without worrying about include order shenanigans. I would just be worried about for example having that type replace the entire map and machinery and force the user to re-implement it - I want exactly what the context vars do, just with a lil' interception ;)

A little late to the party, but I think the best way to solve this is by having a mixin on top of the context. Here's the best I could get so far.
The idea is that you create a "locked context" on each thread and use it instead of the regular context. This context would be lock the given mutex on each call to emplace/etc. In code it would look something like this:
entt::registry registry;

my::mutex ctx_mutex;
entt::locked_context ctx{registry.ctx(), ctx_mutex};

// use ctx just like you would before

Makes sense to me, and looks better than the mixin solution.

Yeah, sure thing. We can also make it a trait actually 🤔 not sure it's worth it though, but I don't like much to dirty the class declaration.

If this becomes a trait, I think the trait should be passed to the registry as a template parameter. This way you can pick a custom context for every registry, which might be useful for "blueprint registries" (ie. having only blueprint entities). The code would then look something like the following:
namespace entt {
    struct default_registry_traits {
        using entity_type = entity;

        template <typename RegistryAlloc>
        using context_type = default_context<RegistryAlloc>;
    };

    template <
        typename Traits, // it's a trait now
        typename Allocator = std::allocator<typename Traits::entity_type>>
    class basic_registry;
}

struct my_traits : entt::default_registry_traits {
    using entity_type = uint64_t;
};

The reason why the Allocator is not specified in the traits are as following:

To break the least amount of code possible. If it were in the traits, the registry would now have one template parameter as opposed to two, which would break code already using a custom allocator.
The allocator depends on the entity type, and having to specify it on every specialization can be bug prone. Consider the previous example again:

struct my_traits : entt::default_registry_traits {
    using entity_type = uint64_t;
    // oops, using the allocator for entt::entity...
};

The other option is to use something on the line of custom data in meta. That is:
auto &elem = registry.ctx<my_context_type>();
elem.emplace<my_type>(42, 'c');

Where the current type is also the default type. This means that registry.ctx() is also backward compatible.
Internally, we only need to allocate the context object and store it with a type id, to debug-assert on use if needed.
This way, one can decide to use a different context type and the whole thing is imho even cleaner than the traits machinery.
Does it make sense? Granted, accessing the context suffers from an extra indirection here, but is it a problem anyway? I don't see the registry context as something used on hot paths in general, it wouldn't make much sense imho.

I keep saying it again and again, but I like this idea better than the traits. The only thing worth to discuss is the lifetime of the context. Given that now the context type will be dynamic internally, there could be an option to change it at runtime, similar to custom data in meta (eg. the context is initially an int variable but then you set it to be a string). I would argue against that, as it feels more natural to reason about code when the type doesn't change and there is no benefit to resetting the context to a different type/value (I think).
If one truly needs the feature, it can easily be emulated by making the context represent several states within the type:
struct context final
{
    // represent states in one type, instead of having separate types of contexts
    bool is_state_1() const;
    bool is_state_2() const;
    // ...

private:
    // internal data
};

However, this means the only way to set the context value (not modify, just set) is when constructing the registry. To me, the simplest way to do this is to have a basic_any<0u> parameter to the registry's constructor that defaults to the "default-context" value; I don't see a reason to have a templated constructor instead of basic_any.

there could be an option to change it at runtime

Yes and no actually. Imagine the following:
template<typename Type = DefaultType>
auto & ctx() {
    ENTT_ASSERT(context_pair.second == nullptr || context_pair.first == type_id<Type>(), "Mismatch");

    if(context_pair.second == nullptr) {
        context_pair.first = type_id<Type>();
        context_pair.second = std::make_shared<Type>();
    }

    return *static_pointer_cast<Type>(context_pair.second);
}

Where context_pair is something like std::pair<type_info, std::shared_ptr<void>>.
All off the top of my head, I don't think this snippet even compiles, but it should give you an idea of what I mean.
In this case, you can't change the context type at runtime basically, you get an assert and you know it. Once you decide, it's that for the life of the registry. Just... with a lazy type.

I have a slightly different idea. I'm guessing that shared_ptr is used for the type-erased deleter. However, the shared_ptr has some extra overhead in memory (allocating the control block, which stores more than just the object). Also, there will only ever be one copy of the context, so there is no need for shared ownership. Also, I would argue that the pair stores the type_info even if not used (ie. when assertions are disabled). type_id "caches" each type_info into static storage, so I think having a reference instead is probably lighter than holding the type_info itself.
Having said that, I think it's best to drop the pair for a unique_ptr<void, Deleter>. The deleter doubles up as a way to retrieve the type_info of the context when needed, or rather a reference to it. In total, this would need just two pointers to work: the pointer to the context, and the pointer to the deleter, and only one member, the unique_ptr. The changes are minimal and the code inside ctx is cleaner (subjective).
Here's the code.

The deleter doubles up as a way to retrieve the type_info of the context when needed

This is REALLY smart.

Ah! You know why this could be tricky maybe? Because of uses-allocator construction and because we should use the registry allocator to instantiate the context. 🤔

This is REALLY smart.

Thanks, good to hear that. 🙂

I'd probably use a class with an extra member rather than the same operator() function for everything.

That's fine by me. I only did it like this because it's shorter.
Also probably worth mentioning, but I just remembered that delete works with nullptr as well (does nothing), so it's perfectly fine if we remove the pointer check on the vtable. I tested with the same code as earlier, and there was no problem.

Yeah, I'm trying to figure out if we can make allocate_unique a little more flexible. It doesn't worth it though probably.

It's probably better to leave allocate_unique as is. It shouldn't be too hard to sneak an allocator in the code. This should be good enough to add allocator support (potentially free for empty allocators), at least for what we discussed so far.


Add sbt-api-mappings plugin

See https://github.com/http4s/http4s-jetty/pull/10.  Nobody should have to think about this stuff.

Tbh I don't really understand what sbt-api-mappings does. It says it only configures stuff for common Scala libraries, so I don't see how it helps with Jetty. Meanwhile, fixing API mappings is IMHO an upstream problem, and it's unclear to me how this plugin interact with sbt's built-in mechanism for this. E.g. see:

https://github.com/playframework/play-json/pull/756

Pretty sure this rule fixes Jetty.  It takes advantage of all Central dependencies being on javadoc.io.
I don't think "upstream mappings" are a concept outside SBT.  Does anything else set that apiURL property in the POM?
I don't know whether this introduces false positives.  Probably, if SBT doesn't resolve the external sources?  But I got frustrated after I patched Jetty when it then couldn't find the JDK.

 Currently the combination of apiURL with the TypelevelUnidocPlugin is working really nice and I' wouldn't want this to override that :)

apiURL is a publisher setting, that adds metadata to the POM.  It isn't a standard I can find anywhere.
autoAPIMappings is a consumer setting, that respects the metadata created by apiURL.  In an ecosystem that uses the same build tool and the same plugins, this works pretty well!
apiMappings is a way to fix it for libraries that don't participate in this system.  That will be most (all?) of Java.
apiMappingRules is a custom thing introduced by this sbt-api-mappings plugin to generate apiMappings.
I'm surprised that the JDK doesn't work out of the box with vanilla SBT!

I guess we can fix the JDK in sbt-typelevel, manually fix jars in projects if we're scared about false positives on javadoc.io, and call it a day.  But I don't think any of this will disturb unidoc functionality.

Right, I'm specifically staring at the apiMappings setting which is a Map. IIUC sbt populates it when autoAPIMappings is enabled and so does the sbt-api-mappings plugin.
https://github.com/ThoughtWorksInc/sbt-api-mappings/blob/872829939dbf80931b3f670b274fe8f7e394d481/src/main/scala/com/thoughtworks/sbtApiMappings/ApiMappings.scala#L48
So my question is, who gets the final say on what goes in there?
After experimenting with it, empirically it does seem to be doing the right thing. So it's probably fine. Ideally I'd like to understand why though.

If I leave autoAPIMappings alone and set doc / apiMappings, it works on 2.12 but not 2.13.  If I instead set apiMappings , it works on 2.13 but not 2.12.

Yeah, idk, it's these weird interactions that make it hard to know for sure what will or won't change/break. Anyway I opened an upstream PR that ensures sbt-api-mappings settings are used only as a fallback, which I think alleviates my concerns.

https://github.com/ThoughtWorksInc/sbt-api-mappings/pull/117

The 2.12 vs. 2.13 weirdness is that the javadoc linking is baked into 2.13, but not 2.12.  I got it working on http4s-jetty with a pair of simple apiMappings.  If we add the Java one here (or sbt-typelevel), we can conservatively do the Jetty jar in the Jetty project, and not add another plugin dependency.

Sure, seems good to upstream it into sbt-typelevel. What's the magic incantation you came up with? :)

https://github.com/http4s/http4s-jetty/pull/10/files#diff-5634c415cd8c8504fdb973a3ed092300b43c4b8fc1e184f7249eb29a55511f91R76-R99
I'm not sure where to put it in sbt-typelevel.  I don't think it has anything to do with Sonatype.

Thanks. I don't really know either, I guess it can go in the Typelevel settings plugin.


Cookie Support

Cookie support is needed for both requests and responses.
Request:

Provide a cookie property with caching to self.parsed_cookies
optimally, it should just be a dictionary of cookie keys to cookie values, to be used like:

cookie_value = request.cookies['cookie_name']

Response:

Interface could look like:

response.cookies['cookie_name'] = 'cookie_value'
response.cookies.set(name='cookie_name', value='cookie_value', secure=True,\
                     http_only=True, domain='.my-domain.com', path='/my-path', \
                     expires=datetime.now(), max_age=timedelta(days=1))

should update the response.headers value to have 'Set-Cookie'

Ideally, this should update the Set-Cookies header of the response when cookies are set.

This is a little harder than I anticipated, is there an external cookie library you'd consider or should it be re-implemented for this project?

Why just don't use aiohttp realization? Do you found some performance problem with this?
get
set

Ah, nice.  I had no idea there was a cookie parser bundled with Python.  I put in basic cookie handling using http.cookies.SimpleCookie.  Thanks both of you for working on this!


Error while writing image data to FPGA

When write a bit stream .bin file to FPGA via FPGA manager, I met this failure:
[ 1758.505854] fpga_manager fpga0: writing 1x16x16_8bx8b_15_15_18_17_100MHz_8ns_v0_0_0.bin to Xilinx ZynqMP FPGA Manager
[ 1758.588004] fpga_manager fpga0: Error while writing image data to FPGA
[ 1758.597566] fpga_manager fpga0: writing 1x16x16_8bx8b_15_15_18_17_100MHz_8ns_v0_0_0.bin to Xilinx ZynqMP FPGA Manager
[ 1758.680143] fpga_manager fpga0: Error while writing image data to FPGA
I'm using board ZCU104 and pre-built PYNQ image "ZCU104 v2.3 SDCard image" released in https://github.com/Xilinx/PYNQ/releases
I saw a similiar issue #767(https://github.com/Xilinx/PYNQ/issues/767) and it's fixed by a patch. I checked the git log of PYNQ reponsitory and found this patch was already merged in ZCU104 v2.3 image.
I also checked the ZCU104 v2.4 image and had the same issue.
Could somebody help to give some suggestions?
Any comments is appreciated.

Hi, @ted51, any updates on this issue? I got the exact same issue with you. I'm using pre-built PYNQ image "ZCU104 v2.4 SDCard image". Any suggestion is appreciated.

Hi @ted51 and @rainfld, I don't know if you got around this issue, but in any case I need to kindly ask you to redirect your question to our forum: https://discuss.pynq.io
That's the dedicated place to receive support from the community.
The issue tracker here is to report bugs in the codebase.
Closing this now, thanks in advance for your collaboration.


Mark Lint/OrderedMagicComments as unsafe autocorrection

This cop's autocorrection is unsafe because file encoding may change.
# bad

# frozen_string_literal: true
# encoding: ascii
p [''.frozen?, ''.encoding] #=> [true, #<Encoding:UTF-8>]

# good

# encoding: ascii
# frozen_string_literal: true
p [''.frozen?, ''.encoding] #=> [true, #<Encoding:US-ASCII>]

It may be the intended file encoding, but the behavior is changed.

Before submitting the PR make sure the following are checked:

[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Ran bundle exec rake default. It executes all tests and runs RuboCop on its own code.
[x] Added an entry (file) to the changelog folder named {change_type}_{change_description}.md if the new code introduces user-observable changes. See changelog entry format for details.

Hmm, isn't it better to just avoid doing this autocorrection?

It may make sense. However, since RuboCop 0.53, this cop has already been released as safe autocorrection and I haven't seen any negative feedback. I found this problem while implementing #11044.
How about seeing what happens with this change?

I meant not to disable the autocorrect, but to make the current autocorrect algorithm smarter, so it would avoid the problematic case.
But I'd be okay with marking this as unsafe.

I see what you mean, but ideally we'll need to explain this better - e.g. the unsafe behavior happens only if there was a bug in the source code to begin with.


PLAN-926 log you out if you have no roles

When does this occur?

as soon as we become aware that you have no roles.
practically, right now, this is only on login.


Removed weak random number generator by adding new import

Signed-off-by: Pankaj Biradar<EMAIL_ADDRESS>

@akhilerm check if its ok now

Codecov Report

Merging #585 (bc9a36d) into master (2f01a37) will decrease coverage by 0.92%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #585      +/-   ##
==========================================
- Coverage   49.47%   48.55%   -0.93%
==========================================
  Files          73       73
  Lines        3359     3359
==========================================
- Hits         1662     1631      -31
- Misses       1541     1572      +31
  Partials      156      156

Impacted Files
Coverage Δ

cmd/ndm_daemonset/probe/eventhandler.go
40.00% <0.00%> (-10.91%)
:arrow_down:

cmd/ndm_daemonset/probe/addhandler.go
71.13% <0.00%> (-6.19%)
:arrow_down:

cmd/ndm_daemonset/probe/udevprobe.go
62.43% <0.00%> (-3.87%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f01a37...bc9a36d. Read the comment docs.


Fix ember-beta tests, fix ember-cli-babel deprecations

This PR fixes the ember-beta scenario and for the same amazing price also updates dependencies to get rid of deprecation warnings from ember-cli-babel :smile: (ie. same as #10)
Fixing the test scenario is "just" a matter of adding the babel polyfill to the build with the includePolyfill: true option for ember-cli-babel.
Let me know if you have any comments :slightly_smiling_face:

Awesome, thanks for doing this!


Cannot get past Python Errors

Ran this in Parrot (similar issues with Kali)
$python3 RsaCtfTool.py --attack cm_factor --publickey key.pub --private
and I cannot get past all the output errors to get a successful run.
`] Testing key key.pub.
[] Performing cm_factor attack on key.pub.
17%|███████▌                                     | 1/6 [00:01<00:05,  1.10s/it][!] Timeout.
17%|███████▌                                     | 1/6 [00:59<04:59, 60.00s/it]
[!] An exception has occurred during the attack. Please check your inputs.
[!] 'Timer expired'
[!] Traceback (most recent call last):
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/lib/rsa_attack.py", line 314, in attack_single_key
self.priv_key, unciphered = attack_module.attack_wrapper(
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/attacks/abstract_attack.py", line 44, in attack_wrapper
return self.attack(publickeys, cipher, progress)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/attacks/single_key/cm_factor.py", line 23, in attack
sageresult = subprocess.check_output(
File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.9/subprocess.py", line 507, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/lib/python3.9/subprocess.py", line 1134, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python3.9/subprocess.py", line 1981, in _communicate
ready = selector.select(timeout)
File "/usr/lib/python3.9/selectors.py", line 416, in select
fd_event_list = self._selector.poll(timeout)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/lib/utils.py", line 198, in _timeout_handler
raise TimeoutError(self.timeout_message)
lib.utils.TimeoutError: 'Timer expired'
Traceback (most recent call last):
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/RsaCtfTool.py", line 491, in
main()
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/RsaCtfTool.py", line 485, in main
args = run_attacks(args, logger)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/RsaCtfTool.py", line 223, in run_attacks
attackobj.attack_single_key(publickey, selected_attacks)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/lib/rsa_attack.py", line 340, in attack_single_key
self.print_results_details(publickey)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/lib/rsa_attack.py", line 99, in print_results_details
print_results(self.args, publickeyname, self.priv_key, self.unciphered)
File "/home/donald/Hackthebox/WeakRSA/RsaCtfTool/lib/utils.py", line 79, in print_results
if any(
TypeError: any() takes exactly one argument (3 given)
`
I'm not sure what information you need but let me know and I can post OS versions, Python version, etc.
Thank you

It seems that you don't have sage installed

@MarcoMeinardi can you try to workout what is failing here, I think it may be related to the latest push.
Thanks

Yes, that is my bad, I forgot to add parenthesis in the any/all calls, sorry.
Skaft has already fixed it, I'm going to double check it later

Merged and fixed.


[Bug]: When I use vllm to start deepseek-ai/DeepSeek-V2-Lite-Chat model inference, the error "deepseek-ai/DeepSeek-V2-Lite-Chat" is reported. My vllm version is 0.6.3，help me bro

Your current environment

The output of `python collect_env.py`
root@e44482312a56:/app/src# python vllmenv.py
WARNING 12-30 10:07:29 cuda.py:22] You are using a deprecated `pynvml` package. Please install `nvidia-ml-py` instead, and make sure to uninstall `pynvml`. When both of them are installed, `pynvml` will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
  from vllm.version import __version__ as VLLM_VERSION
Collecting environment information...
 My vllm version is 0.6.3，My startup command: CUDA_VISIBLE_DEVICES=1,4,5,6 python -m vllm.entrypoints.openai.api_server --model /workspace/pretrained/ --port 10000 --tensor-parallel-size 4 --gpu-memory- utilization 0.95 --max-model-len 1024 --trust-remote-code --enforce_eager --dtype=half
。help me bro
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Can you show the stack trace?

你能显示堆栈跟踪吗？

error log：
root@e44482312a56:/app# CUDA_VISIBLE_DEVICES=1,4,5,6 python -m vllm.entrypoints.openai.api_server --model /workspace/pretrained/ --port 10000 --tensor-parallel-size 4 --gpu-memory-utilization 0.95 --max-model-len 1024 --trust-remote-code --enforce_eager --dtype=half
WARNING 12-30 10:04:17 cuda.py:22] You are using a deprecated pynvml package. Please install nvidia-ml-py instead, and make sure to uninstall pynvml. When both of them are installed, pynvml will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
INFO 12-30 10:04:19 api_server.py:528] vLLM API server version dev
INFO 12-30 10:04:19 api_server.py:529] args: Namespace(host=None, port=10000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=[''], allowed_methods=[''], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/workspace/pretrained/', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, download_dir=None, load_format='auto', config_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=1024, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=4, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=False, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.95, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=True, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=None, qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False)
INFO 12-30 10:04:19 api_server.py:166] Multiprocessing frontend to use ipc:///tmp/970b190e-9f5e-4416-95f0-4b11cdabc2f0 for IPC Path.
INFO 12-30 10:04:19 api_server.py:179] Started engine process with PID 7268
WARNING 12-30 10:04:19 config.py:1674] Casting torch.bfloat16 to torch.float16.
WARNING 12-30 10:04:20 cuda.py:22] You are using a deprecated pynvml package. Please install nvidia-ml-py instead, and make sure to uninstall pynvml. When both of them are installed, pynvml will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
WARNING 12-30 10:04:22 config.py:1674] Casting torch.bfloat16 to torch.float16.
INFO 12-30 10:04:23 config.py:887] Defaulting to use mp for distributed inference
WARNING 12-30 10:04:23 config.py:380] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 12-30 10:04:26 config.py:887] Defaulting to use mp for distributed inference
WARNING 12-30 10:04:26 config.py:380] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 12-30 10:04:26 llm_engine.py:237] Initializing an LLM engine (vdev) with config: model='/workspace/pretrained/', speculative_config=None, tokenizer='/workspace/pretrained/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=1024, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=4, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=/workspace/pretrained/, use_v2_block_manager=True, num_scheduler_steps=1, chunked_prefill_enabled=False multi_step_stream_outputs=True, enable_prefix_caching=False, use_async_output_proc=False, use_cached_outputs=True, mm_processor_kwargs=None)
WARNING 12-30 10:04:26 multiproc_gpu_executor.py:53] Reducing Torch parallelism from 112 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
INFO 12-30 10:04:26 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
WARNING 12-30 10:04:28 cuda.py:22] You are using a deprecated pynvml package. Please install nvidia-ml-py instead, and make sure to uninstall pynvml. When both of them are installed, pynvml will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
WARNING 12-30 10:04:28 cuda.py:22] You are using a deprecated pynvml package. Please install nvidia-ml-py instead, and make sure to uninstall pynvml. When both of them are installed, pynvml will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
WARNING 12-30 10:04:28 cuda.py:22] You are using a deprecated pynvml package. Please install nvidia-ml-py instead, and make sure to uninstall pynvml. When both of them are installed, pynvml will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
/usr/local/lib/python3.10/dist-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:30 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:30 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:30 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
INFO 12-30 10:04:30 utils.py:1008] Found nccl from library libnccl.so.2
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:30 utils.py:1008] Found nccl from library libnccl.so.2
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:30 utils.py:1008] Found nccl from library libnccl.so.2
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:30 utils.py:1008] Found nccl from library libnccl.so.2
INFO 12-30 10:04:30 pynccl.py:63] vLLM is using nccl==2.20.5
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:30 pynccl.py:63] vLLM is using nccl==2.20.5
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:30 pynccl.py:63] vLLM is using nccl==2.20.5
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:30 pynccl.py:63] vLLM is using nccl==2.20.5
(VllmWorkerProcess pid=7561) WARNING 12-30 10:04:31 custom_all_reduce.py:132] Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs. To silence this warning, specify disable_custom_all_reduce=True explicitly.
(VllmWorkerProcess pid=7562) WARNING 12-30 10:04:31 custom_all_reduce.py:132] Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs. To silence this warning, specify disable_custom_all_reduce=True explicitly.
WARNING 12-30 10:04:31 custom_all_reduce.py:132] Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs. To silence this warning, specify disable_custom_all_reduce=True explicitly.
(VllmWorkerProcess pid=7560) WARNING 12-30 10:04:31 custom_all_reduce.py:132] Custom allreduce is disabled because it's not supported on more than two PCIe-only GPUs. To silence this warning, specify disable_custom_all_reduce=True explicitly.
INFO 12-30 10:04:31 shm_broadcast.py:241] vLLM message queue communication handle: Handle(connect_ip='<IP_ADDRESS>', local_reader_ranks=[1, 2, 3], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x7fbcad4d57e0>, local_subscribe_port=54661, remote_subscribe_port=None)
INFO 12-30 10:04:31 model_runner.py:1060] Starting to load model /workspace/pretrained/...
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:31 model_runner.py:1060] Starting to load model /workspace/pretrained/...
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:31 model_runner.py:1060] Starting to load model /workspace/pretrained/...
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:31 model_runner.py:1060] Starting to load model /workspace/pretrained/...
(VllmWorkerProcess pid=7560) Cache shape torch.Size([163840, 64])
Cache shape torch.Size([163840, 64])
(VllmWorkerProcess pid=7562) Cache shape torch.Size([163840, 64])
(VllmWorkerProcess pid=7561) Cache shape torch.Size([163840, 64])
Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  25% Completed | 1/4 [00:01<00:05,  1.71s/it]
Loading safetensors checkpoint shards:  50% Completed | 2/4 [00:02<00:02,  1.45s/it]
Loading safetensors checkpoint shards:  75% Completed | 3/4 [00:04<00:01,  1.60s/it]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:06<00:00,  1.67s/it]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:06<00:00,  1.63s/it]
INFO 12-30 10:04:37 model_runner.py:1071] Loading model weights took 7.3840 GB
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:37 model_runner.py:1071] Loading model weights took 7.3840 GB
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:38 model_runner.py:1071] Loading model weights took 7.3840 GB
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:38 model_runner.py:1071] Loading model weights took 7.3840 GB
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:38 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl...
INFO 12-30 10:04:38 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl...
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:38 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl...
(VllmWorkerProcess pid=7561) INFO 12-30 10:04:38 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl...
(VllmWorkerProcess pid=7562) INFO 12-30 10:04:38 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl.
INFO 12-30 10:04:38 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl.
(VllmWorkerProcess pid=7560) INFO 12-30 10:04:38 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20241230-100438.pkl.
ERROR 12-30 10:04:38 multiproc_worker_utils.py:117] Worker VllmWorkerProcess pid 7562 died, exit code: -15
INFO 12-30 10:04:38 multiproc_worker_utils.py:121] Killing local vLLM worker processes
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner_base.py", line 116, in _wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1665, in execute_model
hidden_or_intermediate_states = model_executable(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/deepseek_v2.py", line 510, in forward
hidden_states = self.model(input_ids, positions, kv_caches,
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/deepseek_v2.py", line 465, in forward
hidden_states, residual = layer(positions, hidden_states,
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/deepseek_v2.py", line 402, in forward
hidden_states = self.mlp(hidden_states)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/deepseek_v2.py", line 149, in forward
final_hidden_states = self.experts(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/layer.py", line 474, in forward
final_hidden_states = self.quant_method.apply(
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/layer.py", line 78, in apply
return self.forward(x=x,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/custom_op.py", line 16, in forward
return self.forward_method(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/layer.py", line 114, in forward_cuda
return fused_experts(hidden_states=x,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/fused_moe.py", line 510, in fused_experts
config = get_config_func(M)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/fused_moe.py", line 369, in try_get_optimal_moe_config
configs = get_moe_configs(E, N, dtype)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/fused_moe.py", line 310, in get_moe_configs
json_file_name = get_config_file_name(E, N, dtype)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/fused_moe.py", line 291, in get_config_file_name
device_name = current_platform.get_device_name().replace(" ", "")
TypeError: a bytes-like object is required, not 'str'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 392, in run_mp_engine
engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 141, in from_engine_args
return cls(
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 78, in init
self.engine = LLMEngine(*args,
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 349, in init
self._initialize_kv_caches()
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 484, in _initialize_kv_caches
self.model_executor.determine_num_available_blocks())
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/distributed_gpu_executor.py", line 39, in determine_num_available_blocks
num_blocks = self._run_workers("determine_num_available_blocks", )
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/multiproc_gpu_executor.py", line 192, in _run_workers
driver_worker_output = driver_worker_method(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 223, in determine_num_available_blocks
self.model_runner.profile_run()
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1309, in profile_run
self.execute_model(model_input, kv_caches, intermediate_tensors)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner_base.py", line 152, in _wrapper
raise type(err)(
TypeError: Error in model execution (input dumped to /tmp/err_execute_model_input_20241230-100438.pkl): a bytes-like object is required, not 'str'
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 585, in
uvloop.run(run_server(args))
File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 82, in run
return loop.run_until_complete(wrapper())
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 61, in wrapper
return await main
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 552, in run_server
async with build_async_engine_client(args) as engine_client:
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 107, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 194, in build_async_engine_client_from_engine_args
raise RuntimeError(
RuntimeError: Engine process failed to start
root@e44482312a56:/app# /usr/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 3 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
/usr/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
z

You should update your pynvml version.


Add showsEndOfRouteFeedback property

Hi!
This is iOS only change.
I've added showsEndOfRouteFeedback property to allow showing/hiding end of route feedback UI on the iPhone.
Let me know if I missed anything. I specifically did not change anything for Android, because I was not able to find corresponding property in Android Navigation SDK. I found
 override fun onFinalDestinationArrival(enableDetailedFeedbackFlowAfterTbt: Boolean, enableArrivalExperienceFeedback: Boolean)

in the kotlin code (MapboxNavigationView.kt line 176) which seems like the Android equivalent, but I don't know enough Kotlin or Android development to expose it in the bridge.

Hey thank you this is great. I'm going to merge this into a feature branch and see if I can get this to work for android also. Either way I'll get this merged into master and publish a new version within the next day.

I got this implemented on android in this feature branch and merged it into master. I also defaulted showsEndOfRouteFeedback to false for both platforms. I think most people will not want that UI showing up at the end of navigation.
It is published in 1.0.4.

@rossmartin Awesome! Thank you!


[FEATURE] Hide/Unhide unverified assets

Аbstract
It is really only recommended for user safety that they trade assets which have had some level of due dilligence performed on them first. TurtleNetwork utilizes their IDD services to provide Ticker and Authorizations to assets on the TurtleNetwork: Qualified, Gateway, etc
Motivation and Purposes
This is to give users a curated and non cluttered default experience, but they can very easily through a toggle in the settings, under advanced, show all assets.
Specification
User has to already have enabled the advanced features through the check box in settings. Then they will see under the set script button a toggle for assets: "Show ALL assets" (switchie)

How do the assets get verified?

How do the assets get verified?

By sending a ticker request either to TN core team, or another data service provider.
Only the people who allowed this data provider will see this ticker

Closed for now


hacktoberfest-2022

Add this your repo as this can help for dictionary
IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

ADD DESCRIPTION HERE
Fixes #
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[x] New feature (It is strongly recommended to open an issue first)

Tip of the day: if you're going to create a spam hacktoberfest PR, at least name it something other than hacktoberfest-2022


Define generalizing a function in Chapter 2 notebook

In the "Parameters" section where you already have a "%TODO: expand this transition" you use the phrase "it would be better to generalize this function" so it takes the probabilities p1 and p2 as parameters:" and you haven't used this language before in the notebooks or textbook. In the relevant part of the textbook you do say, " The advantage of using parameters is that you can call the same function many times, providing different arguments each time." Perhaps you can explicitly describe this as generalization there and it will make more sense in the notebook.

Good idea.  Done.


[Preview 5] Customize the Blazor WASM auth user

Fixes #17650
https://github.com/dotnet/aspnetcore/pull/20587
@mkArtakMSFT ...
AddAccountClaimsPrincipalFactory is a Preview 5 concept, so let's hold this PR for Preview 5.

Thanks for putting this together, @guardrex .
@javiercn can you please help with this? Thanks!

This is confirmed working locally in my claims transformer code (now a Preview 5 app) for AAD roles and security groups.
@javiercn ... if you want to change anything, shoot me a message here or offline.


Aspera SDK - Token issue when developing Javascript.

What we need

How to create and receive tokens using javascript.
Using curl to create permanent (semi-permanent) tokens instead of creating tokens for each file.

Reason.
Problems when generating disposable tokens.
Below is how to create a token that Ravi told you.
Can be created once using curl.
=====================
About test
node account
node id : test
node pw: test
download.json:
{
"transfer_requests": [
{ "transfer_request": {
"source_root": "/test",
"remote_host": "<IP_ADDRESS>",
"paths": [ { "source": "test2.txt"
}
]
}
}
]
}

curl -i -k -X POST -u test:test -d @C:\Users\knp\Desktop\download.json https://<IP_ADDRESS>:9092/files/download_setup

HTTP/1.1 200 OK
Cache: no-cache
Connection: close
Content-Type: application/json; charset=utf-8

{
"transfer_specs" : [
{
"transfer_spec" : {
"paths" : [
{
"source" : "test.json"
}
],
"source_root" : "/test/",
"destination_root" : "",
"token" : "ATM2_ACscsRUiIj8bJco62NaQ8c5Qohsasyg3mGs68-tURANUygAAF7QbULy3dbO9E3hI9DcaMD_2MTA",
"direction" : "receive",
"cipher" : "aes-128",
"tags" : null,
"rate_policy_allowed" : "fixed",
"rate_policy" : "fair",
"target_rate_kbps" : 10000,
"min_rate_kbps" : 0,
"remote_host" : "<IP_ADDRESS>",
"remote_user" : "test",
"sshfp" : null,
"ssh_port" : 33001,
"fasp_port" : 33001,
"http_fallback" : true,
"http_fallback_port" : 8443
}
}
]
}curl: (56) Failure when receiving data from the peer

JavaScript
function download() {
    const transferSpec = {
      authentication: 'token',
      remote_host: '<IP_ADDRESS>',
      remote_user: 'test',
      source_root: '/test',
      paths: [{
        source: 'test2.txt'
    }],
      token: "ATM2_ACscsRUiIj8bJco62NaQ8c5Qohsasyg3mGs68-tURANUygAAF7QbULy3dbO9E3hI9DcaMD_2MTA",
      destination_root: '/',
      direction: 'receive'
    };
    console.log(transferSpec);
    this.client.startTransfer(transferSpec);
  }

    console.log(transferSpec);
    this.client.startTransfer(transferSpec);
  }

When I tested it as above, the file was sent normally.
========================
Using the above method, each time transmission is performed,
a curl token must be generated.
To do that, we need to know how to create tokens with javascript.
Alternatively, we want to know how to create a permanent (semi-permanent) token using curl.

This is more of a Node API question, but you should be able to call the Node API download_setup or upload_setup endpoints like you would any other API from JavaScript (using fetch or axios for example). The contents of the JSON file would simply be the body of the request (no actual file needed). A new token is required for each download because it depends on the transfer parameters (i.e. source file) and also has a valid time range so there's no concept of a permanent token. The closest thing would probably just be using password authentication which I wouldn't recommend for security reasons.
So retrieving a new token for each transfer is recommended, but I'd suggest you do it on the server side to protect your transfer server secrets. Something like this:
const fetch = require('node-fetch');
const https = require('https');

(async () => {
  const httpsAgent = new https.Agent({
    rejectUnauthorized: false // Quirk on the demo server
  });

  const body = {
    "transfer_requests" : [
    {
       "transfer_request" : {
          "source_root" : "/aspera-test-dir-tiny",
          "remote_host" : "demo.asperasoft.com",
          "paths" : [
            {
                "source" : "200KB.1"
            }
         ]
       }
    }
    ]
  };

  try {
    const response = await fetch('https://demo.asperasoft.com:9092/files/download_setup', {
      method: 'POST',
      body: JSON.stringify(body),
      headers: {
        'Authorization': 'Basic ' + Buffer.from('aspera:demoaspera', 'binary').toString('base64') // Should use environment variables for secrets
      },
      agent: httpsAgent
    });

    const data = await response.json();

    const { transfer_specs } = data;
    console.log(transfer_specs);
  } catch (error) {
    console.log(error);
  }
})();

Is there a way to create tokens with Vanilla JavaScript or JQuery?

There are the following types of tokens that can be used to authorize a transfer with Aspera:

Aspera Token: they start and end with AT and are generated by calling the node API endpoints: /files/<upload|download>_setup : typically NOT called directly by a client application, as it requires node api credentials which should usually not be disclosed on client application. But rather a broker application server does that after intermediate validations. That is the example shown earlier. Those tokens are encrypted/decrypted by the server, and have a limited lifespan (defined in aspera.conf)
Basic token: you can use the node api credentials (either node user or access key) as above as token:
'Basic ' + Buffer.from('user:pass', 'binary').toString('base64')
Bearer token (with also a time limit)

Well, as answered earlier:
either use a basic token , but you expose the node credentials on client
or implement a server application that will call download_setup
example:
https://github.com/laurent-martin/aspera-api-examples/blob/main/httpgw/server.js
This example is for the HTTP gateway, but the server side can be used for Aspera Connect as well..

We are not a node.js environment.
I want to know if there is a way to generate tokens from vanilla JavaScript or Jquery, not node.js environment.

I also want to know if I can permanently transfer the file to docroot with the token issued by curl.

Aspera (generaled by calling _setup API) and Bearer tokens are not permanent. (for security reasons).
If you want a permanent token, use the type "basic token" that you can generate as described in a post above.

Can I ask you about basic token generation?

Well,
It is written in my post previously...
const node_user="abcd";
const node_pass="efgh";
const token='Basic ' + Buffer.from(node_user+':'+node_user, 'binary').toString('base64');
console.log(token);

We are not a node.js environment.
It doesn't work if you insert the sample code you gave me.

ERR_CERT_AUTHORITY_INVALID is occurring, so what should I do?

Well...
This is not specific to Aspera:
For security reason, in a browser it is not possible to call an https endpoint with an invalid certificate (e.g. self signed, which is the default in HSTS).
So you have the following possibilities:

use the HTTP interface on HSTS: port 9091. it is not active by default, refer to hsts manual to see how to activate.
install a valid certificate on your server: it needs to be reach by a valid FQDN for which you have a TLS certificate. Refer to the HSTS manual on how to install.

Thanks to you, I solved the problem.
Thank you so much


msg.target should not replace node.target

Hi again,
This behavior is anoying because it forces us to reset msg.target to undefined after clicking on something in order not to click on something else with target defined in the node later on.
msg.target shouldn't replace node.target
node.target should replace msg.target. This behavior exists in many other librairies.
This is the same for every other parameters
https://github.com/simonradier/node-red-contrib-selenium-wd2/blob/7b8567d345afbf5498983c82544675b231ca9424/src/nodes/node.ts#L71-L72

Hi @MyGg29
Just to be sure, you mean that the parameter value from node configuration should be prioritirized over the parameter value from msg?
I got your point with the issue than a msg parameter value will override all node parameter. My question is, how you would override the node configuration if needed?
Thanks for your feedback,

how you would override the node configuration if needed?

You wouldn't be able to. But it's not really a use case is it ? If you want to do that then put a function block before it with
msg.target = msg.overwritten || "defaultValue"
or something like that.
But right now can't do that

You need to do this :

Hi @MyGg29 (long time sorry...),
I made a new package which switches to this behavior.
Thanks again for your piece of advice,


Add files for VISER

Adding files for VISER.
The school's website is https://www.viser.edu.rs/.

@Stereo8 Pull request merged. Thank you!