AttributeError: 'SmartScraperGraph' object has no attribute 'model_token'

Describe the bug
I just followed the instruction and ran the example and getting the following error
I have verified ollama is running and I can use the models using ollama-webui
I am using following models and those are downloaded and working in webui
ollama/gemma:2b
ollama/nomic-embed-text"

Traceback (most recent call last):
  File "/home/leo/dev/projs/scrapegraph/main.py", line 16, in <module>
    smart_scraper_graph = SmartScraperGraph(
                          ^^^^^^^^^^^^^^^^^^
  File "/home/leo/dev/projs/scrapegraph/.venv/lib/python3.11/site-packages/scrapegraphai/graphs/smart_scraper_graph.py", line 24, in __init__
    super().__init__(prompt, config, source)
  File "/home/leo/dev/projs/scrapegraph/.venv/lib/python3.11/site-packages/scrapegraphai/graphs/abstract_graph.py", line 25, in __init__
    self.graph = self._create_graph()
                 ^^^^^^^^^^^^^^^^^^^^
  File "/home/leo/dev/projs/scrapegraph/.venv/lib/python3.11/site-packages/scrapegraphai/graphs/smart_scraper_graph.py", line 39, in _create_graph
    node_config={"chunk_size": self.model_token}
                               ^^^^^^^^^^^^^^^^
AttributeError: 'SmartScraperGraph' object has no attribute 'model_token'

Process finished with exit code 1

To Reproduce
Steps to reproduce the behavior:

Run Ollama using docker
copy paste the example
install scrapegraphai
See error

Expected behavior
It should work seamlessly
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: Ubuntu 23.10

Hi, we did on the branch, just wait till the deployment is done
The configuration is the following:
`"""
Basic example of scraping pipeline using SmartScraper
"""
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai.utils import prettify_exec_info
************************************************
Define the configuration for the graph
************************************************
graph_config = {
"llm": {
"model": "ollama/gemma",
"temperature": 0,
"format": "json",  # Ollama needs the format to be specified explicitly
# "model_tokens": 2000, # set context length arbitrarily,
"base_url": "http://localhost:11434",  # set ollama URL arbitrarily
},
"embeddings": {
"model": "ollama/nomic-embed-text",
"temperature": 0,
"base_url": "http://localhost:11434",  # set ollama URL arbitrarily
}
}
************************************************
Create the SmartScraperGraph instance and run it
************************************************
smart_scraper_graph = SmartScraperGraph(
prompt="List me all the news with their description.",
# also accepts a string with the already downloaded HTML code
source="https://perinim.github.io/projects",
config=graph_config
)
result = smart_scraper_graph.run()
print(result)
************************************************
Get graph execution info
************************************************
graph_exec_info = smart_scraper_graph.get_execution_info()
print(prettify_exec_info(graph_exec_info))`, When we make the next deploy you can use it, just wait


Won't insert second entity upon multiple fetch

Hi,
I am having trouble populating two different entities. Everything work when I fetch them individually. Howver when I try to fech both of them in the mounted the second one is never inserted. XHR works both of them fetched just not inserted.
import { mapGetters } from "vuex";
import TruckList from "@/models/TruckList";
import Order from "@/models/Order";

export default {
  name: "Test",

  computed: {
    ...mapGetters({
      truckListError: "entities/truck-list/errors"
    }),
    orders() {
      return Order.query()
        .withAll()
        .withAllRecursive()
        .all();
    },
    truckList() {
      return TruckList.query()
        .withAll()
        .withAllRecursive()
        .all();
    }
  },

  mounted() {
    Order.$fetch();
    TruckList.$fetch();
  }
};

The two entities are not in any relationship.
I am new with vuex-orm.
No errors been thrown. Not sure how to debug. Not sure what I am doing wrong.

Found the problem. It was vuex-orm axios plugin making duplicate calls.


Reworking CMake files part 2

Original pull request by Volker Enderlein (Bitbucket: VolkerEnderlein, GitHub: VolkerEnderlein).

This is the second and hopefully last part of the CMake build scripts restructuring.

The projects, src, and src/doc CMake build scripts have been heavily restructured to make the different build options and sections more obviously.
A build script for the AppVeyor CI has been added. (that was the primary reason, why I started the restructuring)
Utility macros and functions have been moved to a newly created sub directory cmake.
Lots of find_package commands have been added to support not only runtime linking (loading) of support libraries but also link time linking. The packages include ZLIB, BZip2, OpenAL, Fontconfig, Freetype, SpiderMonkey, simage, superglu. OpenGL, Threads, and Expat are also looked up and used in finding the headers needed.
Most but not all CMake variables have been properly escaped to support spaces in source, build, and install paths (it's an abomination, but some people prefer using it this way).
The config.h file has been extended by a few previously undefined variables.
Some options have been made dependent from other options.
Two find_package modules have been added for finding local Fontconfig and SpiderMonkey installations (SpiderMonkey is not working now in current (and also older) versions either with runtime linking and link time linking, but that's left for another set of commits)
numerous minor changes to the build scripts.

I checked compilation on Windows MSVC 2015 Win64 and Ubuntu.
@veelo and @ggabbiani Feel free to comment on this PR.
#################
Commits included:

Reworking CMake files.
Created cmake directory for scripts.
Fixed wrong usage of define SPIDERMONKEY_RUNTIME_LINKING.
Added numerous #cmakedefine values in config.
Added link libraries to test executable.
Reworking CMake files part II.
Added AppVeyor script.
Added FindSpiderMonkey.cmake module.
Added Coin logo. Escaping qthelpgenerator path (may contain spaces).
Refactor CMake build script to make settings more obvious.
Enhanced find_package script to find older SpiderMonkey versions.
Moved Coins CMake utility functions and macros to separate file.
Restructuring CMake files.
Documentation update.
More restructuring of CMake files. CMake variables properly escaped.

Original comment by Bastiaan Veelo (Bitbucket: veelo, GitHub: veelo).

Impressive work, looks good to me. Thanks!

Original comment by Giampiero Gabbiani (Bitbucket: ggabbiani, GitHub: ggabbiani).

Hi Volker,
you made a great job, really.
This evening I’ll try it on Fedora and let you know.
Just a few consideration:

cpack - we should take a decision about, the cpack.d was though for avoiding package maintenance on each project. Right now is missing Windows support, if you or anyone with Windows skill  could add the Windows part in the dropin we could remove the cpack part from CMake reducing a little the maintenance effort over the collection. If not I can remove cpack.d either, In any case  I would not mantain both.
after having examinated your work with dthe SoGui part for SoQt , I start working on a similar concept for the documentation too: right now each component of the collection has its own documentation part with different output html formatting. I was thinking to use a sub repo to be applied to each component of the collection in order to have a uniform look for the documentation all over the collection. What do you think about?

Best regards
Giampiero

Original comment by Volker Enderlein (Bitbucket: VolkerEnderlein, GitHub: VolkerEnderlein).

Unfortunately I made the pull request to a branch, that is a sub branch of pull request #301. I tried to merge these pull requests but there is no merge button shown on the UI. Am I missing something or do I have not enough privileges to merge into master?

Original comment by Giampiero Gabbiani (Bitbucket: ggabbiani, GitHub: ggabbiani).

Same for me Volker.

Original comment by Volker Enderlein (Bitbucket: VolkerEnderlein, GitHub: VolkerEnderlein).

According to your question/proposals Giampiero.
I fully agree with you not to maintain both packaging strategies. I will have a look into the cpack.d packaging part and try to come up with a proposal/solution for Windows.
For the documentation part I agree with you to synchronize the settings, but I am not quite sure about the subrepo. If it fits well, lets do it this way. Over the last few hours I fixed some of the documentation issues that popped up when building with DOXYGEN_WARN_IF_UNDOCUMENTED set to NO. That's the next commit I am just preparing. Still lots of images are missing, maybe Bastiaan knows where they are located.
Cheers, Volker

Original comment by Bastiaan Veelo (Bitbucket: veelo, GitHub: veelo).

Merge buttons were there for me, so I pressed them. I just checked and you should have rights to write to and create repositories, I would guess that includes merge rights. Let’s see how it is for following PRs, and whether we should review some settings.

Original comment by Bastiaan Veelo (Bitbucket: veelo, GitHub: veelo).

Are images missing here as well? https://coin3d.bitbucket.io/Coin/
Roy generated those docs I believe, and there was an issue with images IIRC. But I don’t know whether it was just a matter of adjusting paths or recreating them.
Maybe they can be recovered from archive.org, for example https://web.archive.org/web/20110116032230/http://www.coin3d.org:80/doc (I just picked a random date).

Original comment by Volker Enderlein (Bitbucket: VolkerEnderlein, GitHub: VolkerEnderlein).

Yes  they are missing as well, e.g. in https://coin3d.bitbucket.io/Coin/classSoScale2Dragger.html
I try to find them in the wayback machine :)

Original comment by Volker Enderlein (Bitbucket: VolkerEnderlein, GitHub: VolkerEnderlein).

@veelo Many thanks for merging the PRs.
When I had a look onto the "Coin" Overview page I saw that I only have access level "Read". That might be the problem for the missing merge capabilities.
@ggabbiani can you have a look too, what your access level is?
Cheers, Volker

Original comment by Giampiero Gabbiani (Bitbucket: ggabbiani, GitHub: ggabbiani).

Yes, same for me the access rights are ‘read’ even if in the history we both should be set as ‘creator’. Actually I’m the same situation for the PR on SoQt.

Regards
Giampiero


chore: Don't run component tests with pr_check.sh

This makes sure pr_check does not trigger component tests with the npm run verify script.

LGTM

:tada: This PR is included in version 1.8.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Intellisense does not load

Type: Bug
VS Code Intellisense will not load in Nuxt3 project.  Have increased memory limits, diasbled and re-enabled  extensions in VS Code to no avail.  Development is being hampered as no intellisense is avialble to use.
VS Code version: Code 1.89.1 (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:13:33.891Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item
Value

CPUs
12th Gen Intel(R) Core(TM) i7-1255U (12 x 2611)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
15.69GB (1.83GB free)

Process Argv
--crash-reporter-id 5282800d-c98e-4333-b69b-e17b6f282b9e

Screen Reader
no

VM
0%

Extensions (7)

Extension
Author (truncated)
Version

vscode-tailwindcss
bra
0.10.5

vscode-html-css
ecm
2.0.9

auto-rename-tag
for
0.1.10

git-graph
mhu
1.30.0

vscode-docker
ms-
1.29.1

prisma
Pri
5.14.0

volar
Vue
1.8.27

A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
pythoncet0:30885854
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30995554
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
chatpanelc:31048052
dsvsc021:30996838
bdiig495:31013172
pythoncenvptcf:31049071
a69g1124:31058053
dvdeprecation:31052039
pythonprt:31056678
dwnewjupyter:31046869
26j00206:31048877

Is it in .vue files? If yes - you should start with Volar extension's repo, as it is responsible for that files support

This issue is caused by an extension, please file it with the repository (or contact) the extension has linked in its overview in VS Code or the marketplace for VS Code. See also our issue reporting guidelines. If you don't know which extension is causing the problem, you can run Help: Start extension bisect from the command palette (F1) to help identify the problem extension.
Happy Coding!


Re-Implement base package ui in React

This issue is for me to track progress of the rewrite on the refactor-to-js branch

as i stated some time before... anything i can help in the rewrite, i'm open to help, i really use this project a lot


Add support for generating OVN client certificates

We currently generate server-specific certificates for OVN, but those are meant for use by OVN services.
It's currently being slightly abused by Incus for its own connection to OVN when it should instead be using a separate client certificate for that.
We should add an ovn_clients config key listing a list of names for which we should generate client certificates.

And extend that to OVN IC with ovn_ic_clients.


Maintenance: prettier is running twice

Affected Packages
N/A, the problem affects only repository maintenance
Problem
Right now, during lint task, prettier is running twice:
https://github.com/changesets/changesets/blob/9956976b7cfee85cc79488476dc5455e76370caa/package.json#L11

by lint:eslint task, which run eslint configured with prettier plugin and the associated rule:
https://github.com/changesets/changesets/blob/9956976b7cfee85cc79488476dc5455e76370caa/.eslintrc#L2-L6
https://github.com/changesets/changesets/blob/9956976b7cfee85cc79488476dc5455e76370caa/.eslintrc#L10
as part of lint:prettier script, which runes directly prettier
https://github.com/changesets/changesets/blob/9956976b7cfee85cc79488476dc5455e76370caa/package.json#L13

[!NOTE]
Removing prettier from eslint brough down lint task from 9s to 6s on my machine.
The result has been obtained by turning off prettier/prettier rule from .eslintrc

Proposed solutions
1. Separate format task from lint task

Remove eslint-plugin-prettier plugin (keep eslint-config-prettier to disable formatting rules)
rename lint:prettier script to format
rename prettier script to format:fix
run format script alongside lint script or in a separate job in CI

2. Remove redundant task
As an alternative I can simply remove lint:prettier script and let format to be part of lint script through eslint

[!NOTE]
If you are interested I can quickly setup a PR with the solution you prefer.

Follow up
As part of the v3 tasks  I see that you plan to update project dependencies.
There is nothing specific about dev dependencies, but, if you wish I can try to update  eslint to v9,
in addition I would also consider to enable some type checking rules (after upgrading to v9).
Let me know if you are interested in discussing this further (in a separated issue or whatever place you prefer 😆).

I'm fine with removing Prettier from ESLint configs etc but we need to make sure that all stylistic rules are turned off in ESLint so they won't conflict with Prettier's formatting.

we need to make sure that all stylistic rules are turned off in ESLint so they won't conflict with Prettier's formatting.

Sure, that's why I proposed to keep eslint-config-prettier (it can be used without plugin and rules),
even if eslint formatting rules are deprecated: https://eslint.org/blog/2023/10/deprecating-formatting-rules/.
I'll open a PR later today.


[BUG]: Fixed navbar and darkmode issue

Closes #538

Description
Hello @cleveranu ,
I have fixed the navbar and the darkmode issue bug which is the major bug for the website because website is not showing the navbar.So I have fixed the issue and it's now wokring properly you can review it and it is super ready to merge. 👍🏻

Fixed Issue Video
Screencast from 04-08-23 10:25:26 PM IST.webm

Checklist

[x] My code adheres to the established style guidelines of the project.
[x] I have included comments in areas that may be difficult to understand.
[x] My changes have not introduced any new warnings.
[x] I have conducted a self-review of my code.

@cleveranu can you please review this.

@SyedImtiyaz-1 Hi, first of all not adding  a navbar was the essence of the design of this page and not an "issue" per say.
Kindly fix the issues mentioned in review.

@Shreyaar12 Okay.

@Shreyaar12 Please add hosted link in the about section it is easier for the contributor for quick checks


Wrong querySession response_type parameter

https://github.com/IdentityModel/oidc-client-js/blob/9bf375464c9173f26c0a19901fc18f4d2db2de1d/src/UserManager.js#L220
This line should be:
args.response_type = args.response_type || this.settings.response_type

Yes, but a lot of providers defines response type as a list of choices:

id_token
id_token token
and so on to define Implicit Flow, Code Flow or Hybrid Flow

When you define client with Implicit Flow id_token token then IdP returns
{
  "name":"ErrorResponse",
  "error":"invalid_request",
  "error_description":"The request is otherwise malformed"
}

And I have question about this on couple lines bellow:
if (signinResponse.session_state && signinResponse.profile.sub && signinResponse.profile.sid)

What is sid ? I can't find this field in OpenID Core spec and my IdP does not return this value either.

All set on this issue -- can we close?

I haven't received response

When you define client with Implicit Flow id_token token but you send only id_token then IdP returns

I don't know. This sounds like some issue with your token server.

What is sid ? I can't find this field in OpenID Core spec and my IdP does not return this value either.

It's in the session specs: http://openid.net/connect/


Update GitHub Actions versions in README

Update to match those used in the current workflows here.

Thank you!


Interaction crash when interactions are too close to eachother

https://github.com/Phazorknight/Cogito/assets/57206297/d93cc306-35ea-4f8a-abce-0dfa8b830b37

When picking up the battery, it is too close to the drawer, from what i understand its causing the interaction on the drawer to be free'd, so when trying to close the drawer after, it causes a crash.

This is odd, so far I wasn't able to reproduce this.
I see you changed the default interact actions, can you confirm which ones you have on the drawer and the battery when this error occurs?
I do know one or two issues with the interaction raycast check that can be improved that might fix this error, so I'll look into that as well.

Ok, try the latest commit from main and see if this might fix your issue: https://github.com/Phazorknight/Cogito/commit/e8198c26ad42f419f4637c0e65dc248a80082ca3


Datetime picker

Is your feature request related to a problem? Please describe
I'd like users to specify a time range by defining two datetimes. The beginning of the time range cannot lie in the past and the end (obviously) must be later than the beginning. Currently, if a minimum time for the time picker is specified this limits the time for all days even those in the future that would specify a valid end time.
Describe the solution you'd like
Currently, separate date and time pickers are available. A combined datetime picker has been suggested before here: https://github.com/h2oai/wave/issues/1537. My motivation for the request is different then for that issue which is why I created a new one.
I'd like either a combined datetime picker that takes a min and max limit as datetime. Alternatively, my problem would be solved if the time picker takes a datetime or timestamp as limit (but that does not seem like a good solution).
Describe alternatives you've considered
The alternative for my use case would be to remove min/max limiters and add a validation step that prevents form submission for invalid start/stop inputs. But UI limitation would be easier to use.
Additional context
None

Is your feature request the same as https://github.com/h2oai/wave/issues/1012?

I would also need time specification, not just date range

I don't have an example. I kind of thought I had seen something like this on booking pages for trains and flights but if I look for it now it is indeed separate pickers.
I did not think it was such a niche use case :sweat_smile: but maybe it is.
Anyway, I thought I'd request it as a nice-to-have and maybe it would be also useful for others. For now, I "solved" it for my use-case by adding the validation step.

Gotcha, sounds reasonable. Will leave this open for now. Since the component would be fairly complex, I will backseat it for now, but happy to accept community PRs in case of interest.


How to logout from script?

Whenever I use the script, the script leaves me signed in, is there any function such as mega.logout()?

There does not appear to be one, but I added one to the Mega class like so:
def logout(self):
        return self._api_request({'a': 'sml'})


Application fails when specified a non-zero disk resource of mounted disk

I have tried running some apps on Marathon. Apps run only on root disk of Mesos. When a mounted disk is specified in resources file(/etc/mesos-slave/resources) of mesos agent, apps didn't run (are in waiting state forever). It seems mounted disk resource is not offered to Marathon framework. Is multiple disk feature supported in Marathon ?

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-4662. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-4662. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


Investigate / review changes in results for numpy versions > 2.0

Numpy 2.1 breaks the long-term integration tests, the eigenvalues and eigenvectors have changed for some test cases.

It seems the eigenvalues and -vectors are sorting only in a different sequence. The flutter speed and the curves look unchanged in the plots.


Dpl 996 pacbio pool create, pool stores to pinia

pacbioPoolCreate, pacbioPools migration to Pinia
tags and tagsets moved to root store

This is a good start.
I think now the service is ready it would be useful to test how this behaves with used_aliquots vs libraries. One caveat I can see is used_aliquots don't have a pacbio_request_id they have a source_id. We could edit used_aliquots when we retrieve them to set pacbio_request_id as source_id.
I think we also need to consider if/how we feature flag used_aliquots vs libraries. If we feature flag we will need to change how we retrieve pools, update and create pools.

Thank you. I think setting source_id can be done as part of store action.

Looks good 👌 May be a few things we need to tweak when we come to implementation but this seems as good as it can be considering the unknowns.

Thank you very much for your feedback.
Yes, sure. We can always come back to this and modify (if required) as part of component stories I hope.


Using a 'web server aware' Node library, with JavaScriptServices / NodeServices (React SPA)

(Although this topic specifically mentions i18next, this could equally apply to other node packages that typically expect to have access to a Node web server such as Express, and some guidance would be most helpful.)
Scenario:
I wish to integrate the Node internationalization library i18next to provide language detection (initially browser accept-language headers in the request, or url segment such as /en-GB/), client side language selection (drop down), client side persistence of selected language (cookie / local storage / Redux), caching of translation data (local storage / Redux), translation of text client side and most importantly in the pre-rendered server side environment.
Usual approach:
The normal way to do this with a fully isomorphic JavaScript app would be to use a plugin package (in this case i18next-express-middleware), that has access to the server-side request and response, to perform language detection by accessing the request context - example server.js here.
JavaScriptServices / NodeServices approach:
The actual loading of locales, and translation of text within my React components, can be achieved as normal once the i18next instance is initialised (tested this by specifying a default language).
The owner of the i18next project described the process of language detection and instance initialization as follows:

Regarding language detection (or recover via cookie, session, what ever) this has to happen inside the request code => as a sample express middleware does this on every request using the middleware: https://github.com/i18next/i18next-express-middleware/blob/master/src/index.js#L23
From there you:
a) create a fixedT function to use inside request: https://www.i18next.com/api.html#getfixedt
b) create a new clone of main instance and change language on that: https://www.i18next.com/api.html#cloneinstance --> https://github.com/i18next/i18next-express-middleware/blob/master/src/index.js#L13 (on line 30 is the changeLanguage call)

However, it seems to me that NodeServices execution context has no knowledge of the web environment itself, other than the data that is passed to it via the asp-prerender-data tag helper.
So, it also seems to me that any examination of the web request has to be performed in the .Net Core pipeline, with relevant parameters determined.
This would then be passed from the controller (e.g. Home/Index) to the corresponding view that contains asp-prerender-module tag and use asp-prerender-data to pass along these parameters, which can then be accessed in the server-boot.ts module. Here, I would use them to initialise the i18next instance, and allow its provider to supply locales to my components as normal, as they are rendered by NodeServices / WebPack.
Then, I should pass data - the language detected via ASP.Net Core, the locale json data loaded from the server filesystem - back to the client via the globals return 'type', as is currently done with initial state for Redux. This would then be attached the data to the window object on the client side, and could be persisted into local storage to allow i18next to take over and work as normal from then on.
Questions:

Is my understanding of the Kestrel > AspNet Core > NodeServices flow, and that NodeServices is essentially a non-web aware Node context, correct?

The advice from the owner of i18next to 'clone the main instance of i18next, and change the language on that', is to ensure each request receives its own instance, with its own language detection result. I am presuming that this is only required due to Node / Express, and that in the JavaScriptServices / NodeServices framework, each request would have its own isolated Node context?

It's not clear to me exactly how NodeServices interacts with WebPack to return transpiled output to the browser, or how that pipeline is arranged. Is there an in-depth explanation somewhere of the full interaction of each component in a typical lifecycle?

Does the team have any guidance in terms of internationalization, in the current JavaScriptServices / NodeServices framework?

Many thanks for any feedback...

Update:
I did find this in HttpNodeInstanceEntryPoint.ts:
const server = http.createServer((req, res) => {
Which indicates that NodeServices might be able to be web aware...?

Closed and opened a simpler issue - this seems too wordy.


[FEATURE] Add more functionality to washboard-ui LatticeService

We need the ability to customize the topic that washboard listens on. At the moment, it uses the wasmbus.ctl.default, but both the topic prefix (wasmbus) and the lattice prefix (default) need to be configurable.
The LatticeSettings component should be able to update both of these settings.

The service has been updated but the washboard-ui doesn't yet make use of the new settings. This one is free to implement for anyone who'd like to contribute.


Add Integration Tests for Ephemeral Volumes

Description
What problem are you trying to solve?
We currently don't have E2E integration tests that test that ephemeral volumes work with Karpenter and that Karpenter respects the CSI volume limits for ephemeral volumes when scheduling. We should add tests to validate this.
Code Pointers

https://github.com/aws/karpenter/blob/main/test/suites/integration/storage_test.go
https://github.com/aws/karpenter-core/blob/main/pkg/controllers/provisioning/scheduling/suite_test.go#L2591

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Closed by #5229


Full SQL Model: update the aggregation queries

Update them with the changes made to incremental.

Closing because not relevant anymore.


How to backup Prometheus' most recent 2 hours of data?

Sorry in advance, this question is probably best suited to a community forum, but I couldn't find one?
I'm looking for some thoughts about whether Thanos has a way, or if the team is interested in developing a way, to make the most recent 2 hours of data collected by Prometheus durable against crashes.
 As I understand it, this does not include the most recent two hours of data, which Prometheus is storing in memory as it decides how to compress and compact it.
If the Prometheus instance crashes or gets OOM killed, or the underlying hardware fails, it seems to me that I'll lose those two hours of data if I'm relying on Thanos's S3 storage for recovery?
I understand that Prometheus has a WAL for this purpose, and can in theory recover from a crash by reading the WAL back into memory. However, there are lots of bugs reported around Prometheus's WAL crash recovery (e.g. https://github.com/prometheus/prometheus/issues/5577, https://github.com/prometheus/prometheus/issues/4603, https://github.com/prometheus/prometheus/issues/4842). Indeed, one of the reasons we have deployed Thanos now is because we lost monitoring data when we accidentally gave Prometheus too many metrics and it started crash-looping with a "WAL torn" error.
If I want the most recent 2 hours of metric data to be durable, is the best option to configure a separate remote-write endpoint, or is there a way I can set up my Thanos to preserve the most recent data too? Or is there another option I should be considering?

Hello there, have you ever considered trying this https://thanos.io/proposals/201812_thanos-remote-receive.md/ out? :smile: Sounds like it solves the problem that you are having.

i think you need to change your Prometheus deployment.
if you want the Prometheus to be highly available you need two instances of it that are scraping the same targets. this will allow you to keep metric collection even in case your Prometheus server has suffered a failure and is down.
this coupled with Thanos and its ability to dedupe the results will allow you to see metrics at any time even if your server is down.
one more point in your current deployment, you will not be able to upgrade your server or change any configuration flag as in both cases you have to take the server down and in your curent deployment the metrics will not be colected during that time.
i suggest you take a look at the improbable reference architecture here:
https://improbable.io/blog/thanos-architecture-at-improbable

Thanks GiedriusS! Yes, I think https://thanos.io/proposals/201812_thanos-remote-receive.md/ would address my concern about preserving the most recent 2 hours of data in the event of failure. To me, it also raises the question of whether the sidecar would still be necessary if we have a remote-write endpoint? I suppose that's a question for the maintainers though.
Hi miclando, thanks for your feedback! I do indeed run two prometheus instances on separate boxes. However, there are correlated failure modes I'm concerned about. In particular, Prometheus will choke if it tries to ingest too many metrics. If we have an emergency and ops starts cycling docker containers much faster than usual, it's possible to suddenly start recording many more metrics and cause both servers to start OOM looping.
Obviously, it would be nice if Prometheus was able to handle overload more gracefully. In the mean-time, spinning up another server with more memory is a relatively quick solution to get scraping working again, but risks losing the data from the old server that isn't yet backed up.
Specific scenarios aside, we will continue to discover previously unknown, or believed to be unlikely, ways in which a couple of boxes running the same software can fail at about the same time - and there's always a possibility of human error. It's a best practice to have the data backed up more durably.

@GiedriusS Is there an implementation of that proposal somewhere that I can use?

Hi @alexf101, my organisation is also facing the same issue of backing up the latest 2 hrs of data in prometheus. Please share how you resolved this issue. We are currently planning to adopt Thanos, with 2 or more instances of Prometheus. Thanks.


Wrong function description in documentation

https://github.com/yiisoft/yii2/blob/master/framework/grid/GridView.php#L347
Returns string (there are void in docs)

Would you like to submit a Pull Request to fix it?


GH-3151: Add events to ICakeEngine to invoke at the end of Setup/Teardown

This allows for any logic required after each section to be executed.
Fixes #3151

@BlythMeister Just to give you some feedback on this one, we discussed this PR internally and given that it introduces a breaking change to the public API (i.e. ICakeEngine) which would impact all extensions, we'll consider it for Cake v2.0.0 later this year, at which time we'll engage with you to understand more about the use-case of these events and test some examples together.

Added a comment to the issues where to discuss the requirements and design first: https://github.com/cake-build/cake/issues/3151#issuecomment-945202748

Yeah perfectly fine with me.  It's been on my to-do list...but work has been super busy!

Who is to resolve the comments?
I'm now not sure what I did originally and what was modified.
If you want to wait for me, I'm not sure what timeline that will be, as I am super busy at work at the moment.

Who is to resolve the comments? I'm now not sure what I did originally and what was modified.
If you want to wait for me, I'm not sure what timeline that will be, as I am super busy at work at the moment.

@BlythMeister That's fine. I took care of resolving the comments

@BlythMeister your changes have been merged, thanks for your contribution :+1:


Collection.import_bulk : halt_on_error defaults to None, not True

This was a nasty gotcha: in the documentation for import_bulk it says "halt_on_error" defaults to True, in fact it defaults to None meaning that import_bulk by default silently ignores errors.
I believe it would be best if it actually defaults to True, but in any case the documentation should not be contradictory.

Fixed in version 4.0.0. Please note that this version is not backward compatible with 3.x and under.


ScrollView child layout must be applied through the contentContainerStyle prop

Version
1.4.6
Reproduction
import styled from 'styled-components/native';

const Scroll = styled.ScrollView`
  align-items: 'center';
`;

this code throws an error
ScrollView child layout (["alignItems"]) must be applied through the contentContainerStyle prop.

Expected Behavior
I'd expect styled components would apply the property to the correct element.
Actual Behavior
I ended up writing
const styles = StyleSheet.create({
  center: {
    alignItems: 'center',
  },
});

<ScrollView contentContainerStyle={styles.center}>
    {children}
</Scroll>

I was able to perform this using the following:
const Scroll = styled.ScrollView.attrs({
  contentContainerStyle: props => {
    return {
      alignItems: 'center',
      justifyContent: 'center'
    }
  }
})``

Yep this is why we added .attrs! Please use v2 and the .attrs helper to add styles to other props.

Bump, Subscribing

If you are using react native with typescript, you can use it like this
 interface ScrollViewProps {   // usually I define here any props passed from component to style } const StyledScrollView = styled.ScrollView.attrs<ScrollViewProps>({   contentContainerStyle: () => css
align-items: center;
, })``;

If you are using react native with typescript, you can use it like this
const StyledScrollView = styled.ScrollView.attrs(() => ({
  contentContainerStyle: {
    alignItems: 'center',
  },
}))``;

Nice one! Why does this one relate to TypeScript only ?

kolkol69's answer is correct because styled-components has now deprecated the earlier method (as in, you should make attrs directly return a function rather than an object)

If you are using react native with typescript, you can use it like this
const StyledScrollView = styled.ScrollView.attrs(() => ({
  contentContainerStyle: {
    alignItems: 'center',
  },
}))``;

To use your method, should I install or import any dependencies ???
if yes what should I do ??
please help!

@dhruvash2u No extra dependencies, it's built into styled-components. What have you tried so far? If it's not working with your current code, could you please share it?

I was using simple react native but when i used native base and placed everything in    it gives me this error.
some one please guide me

I was using simple react native but when i used native base and placed everything in    it gives me this error.
some one please guide me

This worked for me with react-native
const Scroll = styled.ScrollView.attrs((props) => ({
contentContainerStyle: {
flexGrow: 1,
padding: 20,
},
}))    flex: 1;  ;

Typescript :
export const StyledScrollView = styled.ScrollView.attrs((props) => ({
  contentContainerStyle: {
    backgroundColor: "red", // default contentContainerStyle
    padding: 20,
    ...(props.contentContainerStyle as object), // override contentContainerStyle
  },
}))(() => ({
  // scrollview styles
}));


Developer workflow optimizations

Something we need to continue to give some thought to is what the developer experience will be when working on an application. The primary use case is where the developer works on the project locally using whatever build/dev tools the framework provides (e.g. ng), and then only deploys when development is "complete" (for some value of completeness). But there will be times when the developer wants or needs to make changes in a running container for testing purposes. The OpenShift solution for this is typically rsync the working directory on the container, and that may be sufficient. But we should be sure about that and document it.

This is sort of what i was talking about in the meeting today.
Currently if a user builds and deploys their application with this image not using a chain build, then this images "run" phase will serve the build output using the serve module.
Lets take a react app as an example.  If someone is creating a react application locally, they usually do a npm start which starts the app in development mode, which watches for edits and recompiles/reloads the application.  I'm going to call that the dev server.
What if the run phase was able to run this dev server on openshift,  then locally, the user can run oc sync --watch .... which will keep their local code and the remote pod in sync.  which should also trigger the dev server to recompile.
of course the example i just showed was with react,  but angular/ember/....'s dev server could probably be easily be substituted.

@lholmquist agree this sounds like a good use case and workflow to hammer out.


roslaunch ur_modern_driver ur5_bringup.launch  CONNECTION SETUP FAILED

PLEASE HELP!
I am trying to launch:
roslaunch ur_modern_driver ur5_bringup.launch robot_ip:=<IP_ADDRESS> [reverse_port:=REVERSE_PORT]
However am getting the following error:
[ERROR] [1549412928.839559142]: Connection setup failed for <IP_ADDRESS>:30003
[ INFO] [1549412928.839573683]: Setting up connection: <IP_ADDRESS>:30001
[ERROR] [1549412928.839593882]: Connection setup failed for <IP_ADDRESS>:30001
[ WARN] [1549412928.839602715]: Failed to read from stream, reconnecting in 1 seconds...

I am able to ping the robot obtaining this:
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.019 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.032 ms

I have set up following this tutorial http://wiki.ros.org/universal_robot/Tutorials/Getting Started with a Universal Robot and ROS-Industrial
How do I fix this issue?

Following from this I have used a static network and used the IP address from the setup (3.2 of the tutorial) as the IP address of the robot. Is this correct?

@Anastasiamor100 wrote:

PLEASE HELP!

please don't do this. We already know you are looking for help, for otherwise you would not have posted here. Adding such lines to issues doesn't motivate anyone to help you any sooner.
Second: please provide more information on your setup. Right now we don't know anything, apart from that you're trying to start the launch file for a UR5.
So tell us:

which OS you are using
which version of ROS you are using
are you running things in a virtual machine (VMWare, VirtualBox, Parallels)?
which version of ur_modern_driver you are using (master branch? kinetic-devel?)
which controller your robot has (CB2, CB3, e-series)
which version of the system software/Polyscope it is running (1.8.x, 3.x or a 5.x variant)

Following from this I have used a static network and used the IP address from the setup (3.2 of the tutorial) as the IP address of the robot. Is this correct?

Only if the host you are using to run ur_modern_driver is also part of the 192.168.1.x network.

I am trying to launch:
roslaunch ur_modern_driver ur5_bringup.launch robot_ip:=<IP_ADDRESS> [reverse_port:=REVERSE_PORT]

Are you literally typing [reverse_port:=REVERSE_PORT]?
If you are, just don't specify the reverse_port argument, ie start the launch file with just:
roslaunch --screen ur_modern_driver ur5_bringup.launch robot_ip:=<IP_ADDRESS>

If that doesn't work, copy-paste all output into a comment here and we can try to resolve your problem.

And I'm going to assume that this is not an issue with the pkg in this repository, but a user configuration one, so I'll close this ticket.
That doesn't mean we can't solve your configuration issue, so please feel free to keep commenting on the issue.


[Core] Implement disagg prefill by StatelessProcessGroup

A light-weight implementation of disaggregated prefill. I switched from PR #8498 to here in order to fix DCO issues.

Thank you @coolkp , I will fix this example in the next PR.

Yes this is the right way. But please constraint GPU util to a smaller value -- current implementation buffers KV cache in GPU and it is not included in GPU util, so the GPU util needs to be smaller to leave space for KV cache buffer

Hello. @KuntaiDu  Shortly after a successful invocation to the proxy. I got these errors from the vllm instances.  Anything goes wrong?
[rank0]:[W1202 17:11:14.631061365 socket.cpp:487] [c10d] waitForInput: socket SocketImpl(fd=125, addr=[::ffff:<IP_ADDRESS>]:50228, remote=[::ffff:<IP_ADDRESS>]:14580) timed out after 300000ms
ERROR 12-02 17:11:14 pynccl_pipe.py:261] Encountering exception in KV receiving thread
ERROR 12-02 17:11:14 pynccl_pipe.py:262] wait timeout after 300000ms, keys: /send_to/0/4
ERROR 12-02 17:11:14 pynccl_pipe.py:263] My device: cpu
Traceback (most recent call last):
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor
tensor = future.result()
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl
metadata = self._recv_metadata()
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata
return self.group.recv_obj(self.target_rank_for_recv)
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj
self.store.get(
torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4
Exception in thread Thread-3 (drop_select_handler):
Traceback (most recent call last):
File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 177, in drop_select_handler
raise e
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 132, in drop_select_handler
signal = self.signal_pipe.recv_tensor()
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 266, in recv_tensor
raise e
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor
tensor = future.result()
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl
metadata = self._recv_metadata()
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata
return self.group.recv_obj(self.target_rank_for_recv)
File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj
self.store.get(
torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4

Hello. @KuntaiDu Shortly after a successful invocation to the proxy. I got these errors from the vllm instances. And the second invocation will be blocked. Anything goes wrong? or any parameter should be configured. Thx!
[rank0]:[W1202 17:11:14.631061365 socket.cpp:487] [c10d] waitForInput: socket SocketImpl(fd=125, addr=[::ffff:<IP_ADDRESS>]:50228, remote=[::ffff:<IP_ADDRESS>]:14580) timed out after 300000ms ERROR 12-02 17:11:14 pynccl_pipe.py:261] Encountering exception in KV receiving thread ERROR 12-02 17:11:14 pynccl_pipe.py:262] wait timeout after 300000ms, keys: /send_to/0/4 ERROR 12-02 17:11:14 pynccl_pipe.py:263] My device: cpu Traceback (most recent call last): File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor tensor = future.result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl metadata = self._recv_metadata() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata return self.group.recv_obj(self.target_rank_for_recv) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj self.store.get( torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4 Exception in thread Thread-3 (drop_select_handler): Traceback (most recent call last): File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 177, in drop_select_handler raise e File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 132, in drop_select_handler signal = self.signal_pipe.recv_tensor() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 266, in recv_tensor raise e File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor tensor = future.result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl metadata = self._recv_metadata() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata return self.group.recv_obj(self.target_rank_for_recv) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj self.store.get( torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4

Currently, PyNcclPipe does not implement heartbeats, so the connection will be closed if no request is sent within 5 minutes.

Hello. @KuntaiDu Shortly after a successful invocation to the proxy. I got these errors from the vllm instances. And the second invocation will be blocked. Anything goes wrong? or any parameter should be configured. Thx!
[rank0]:[W1202 17:11:14.631061365 socket.cpp:487] [c10d] waitForInput: socket SocketImpl(fd=125, addr=[::ffff:<IP_ADDRESS>]:50228, remote=[::ffff:<IP_ADDRESS>]:14580) timed out after 300000ms ERROR 12-02 17:11:14 pynccl_pipe.py:261] Encountering exception in KV receiving thread ERROR 12-02 17:11:14 pynccl_pipe.py:262] wait timeout after 300000ms, keys: /send_to/0/4 ERROR 12-02 17:11:14 pynccl_pipe.py:263] My device: cpu Traceback (most recent call last): File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor tensor = future.result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl metadata = self._recv_metadata() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata return self.group.recv_obj(self.target_rank_for_recv) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj self.store.get( torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4 Exception in thread Thread-3 (drop_select_handler): Traceback (most recent call last): File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/data/miniconda3/envs/will/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 177, in drop_select_handler raise e File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_lookup_buffer/simple_buffer.py", line 132, in drop_select_handler signal = self.signal_pipe.recv_tensor() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 266, in recv_tensor raise e File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 259, in recv_tensor tensor = future.result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 458, in result return self.__get_result() File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result raise self._exception File "/data/miniconda3/envs/will/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 189, in _recv_impl metadata = self._recv_metadata() File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/kv_transfer/kv_pipe/pynccl_pipe.py", line 164, in _recv_metadata return self.group.recv_obj(self.target_rank_for_recv) File "/data/miniconda3/envs/will/lib/python3.10/site-packages/vllm/distributed/utils.py", line 148, in recv_obj self.store.get( torch.distributed.DistStoreError: wait timeout after 300000ms, keys: /send_to/0/4

Currently, PyNcclPipe does not implement heartbeats, so the connection will be closed if no request is sent within 5 minutes.

Is there a workaround to solve this timeout. Or how to modify the 5min timeout. Thanks!

Is there a workaround to solve this timeout. Or how to modify the 5min timeout. Thanks!

This is a known issue, it will be addressed in the future PR by @KuntaiDu . If you need a quick workaround, you can modify disagg_prefill_proxy_server.py to send a shadow request every 4 min through apscheduler.

Hello, I encountered the following issue while running the decomposition reasoning on the 'mian' branch:
ValueError: not enough values to unpack (expected 4, got 2).
The actual real kvcache shape is  “kv_cache[0] shape torch.Size([2162, 81920])”
INFO 12-03 14:31:48 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241203-143148.pkl...
INFO 12-03 14:31:48 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20241203-143148.pkl.
ERROR 12-03 14:31:48 engine.py:135] ValueError('Error in model execution (input dumped to /tmp/err_execute_model_input_20241203-143148.pkl): not enough values to unpack (expected 4, got 2)')
ERROR 12-03 14:31:48 engine.py:135] Traceback (most recent call last):
ERROR 12-03 14:31:48 engine.py:135]   File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner_base.py", line 116, in _wrapper
ERROR 12-03 14:31:48 engine.py:135]     return func(*args, **kwargs)
ERROR 12-03 14:31:48 engine.py:135]   File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner.py", line 1718, in execute_model
ERROR 12-03 14:31:48 engine.py:135]     get_kv_transfer_group().send_kv_caches_and_hidden_states(
ERROR 12-03 14:31:48 engine.py:135]   File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 60, in send_kv_caches_and_hidden_states
ERROR 12-03 14:31:48 engine.py:135]     self.connector.send_kv_caches_and_hidden_states(
ERROR 12-03 14:31:48 engine.py:135]   File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_connector/simple_connector.py", line 134, in send_kv_caches_and_hidden_states
ERROR 12-03 14:31:48 engine.py:135]     _, _, num_heads, head_size = kv_cache[0].shape
ERROR 12-03 14:31:48 engine.py:135] ValueError: not enough values to unpack (expected 4, got 2)
ERROR 12-03 14:31:48 engine.py:135]
My startup command is:
CUDA_VISIBLE_DEVICES=3 nohup nohup python3
-m vllm.entrypoints.openai.api_server
--model $model
--port 8100
--max-model-len 1000
--gpu-memory-utilization 0.7
--kv-transfer-config  '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4  nohup python3
-m vllm.entrypoints.openai.api_server
--model $model
--port 8200
--max-model-len 1000
--gpu-memory-utilization 0.7
--kv-transfer-config  '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}'  > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &
My startup command is:
CUDA_VISIBLE_DEVICES=3 nohup nohup python3
-m vllm.entrypoints.openai.api_server
--model $model
--port 8100
--max-model-len 1000
--gpu-memory-utilization 0.7
--kv-transfer-config  '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4  nohup python3
-m vllm.entrypoints.openai.api_server
--model $model
--port 8200
--max-model-len 1000
--gpu-memory-utilization 0.7
--kv-transfer-config  '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}'  > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &

Hi @liweiqing1997 , currently I only tested Llama-style model. What kind of model are you using?

Hello, I encountered the following issue while running the decomposition reasoning on the 'mian' branch: ValueError: not enough values to unpack (expected 4, got 2).
The actual real kvcache shape is “kv_cache[0] shape torch.Size([2162, 81920])”
INFO 12-03 14:31:48 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241203-143148.pkl... INFO 12-03 14:31:48 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20241203-143148.pkl. ERROR 12-03 14:31:48 engine.py:135] ValueError('Error in model execution (input dumped to /tmp/err_execute_model_input_20241203-143148.pkl): not enough values to unpack (expected 4, got 2)') ERROR 12-03 14:31:48 engine.py:135] Traceback (most recent call last): ERROR 12-03 14:31:48 engine.py:135] File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner_base.py", line 116, in _wrapper ERROR 12-03 14:31:48 engine.py:135] return func(*args, **kwargs) ERROR 12-03 14:31:48 engine.py:135] File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner.py", line 1718, in execute_model ERROR 12-03 14:31:48 engine.py:135] get_kv_transfer_group().send_kv_caches_and_hidden_states( ERROR 12-03 14:31:48 engine.py:135] File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 60, in send_kv_caches_and_hidden_states ERROR 12-03 14:31:48 engine.py:135] self.connector.send_kv_caches_and_hidden_states( ERROR 12-03 14:31:48 engine.py:135] File "/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_connector/simple_connector.py", line 134, in send_kv_caches_and_hidden_states ERROR 12-03 14:31:48 engine.py:135] _, _, num_heads, head_size = kv_cache[0].shape ERROR 12-03 14:31:48 engine.py:135] ValueError: not enough values to unpack (expected 4, got 2) ERROR 12-03 14:31:48 engine.py:135]
My startup command is:
CUDA_VISIBLE_DEVICES=3 nohup nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8100 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4 nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8200 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}' > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &
My startup command is:
CUDA_VISIBLE_DEVICES=3 nohup nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8100 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4 nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8200 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}' > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &

I guess you are using a GPU card with Volta or Turing architecture? I have found this problem in the older version of this PR. @KuntaiDu If you don't have bandwidth, I can propose a PR to fix this.

Hi @liweiqing1997 , currently I only tested Llama-style model. What kind of model are you using?

I am testing the Qwen 1.5B chat. Previously, I tested a version that had not been merged into the vllm/main branch, and it ran successfully. However, the main branch version does not work. I'm not sure if any changes were made or if there is an issue with my settings.

BTW feel free to also comment in disaggregated prefill roadmap (#10818)

您好，我在“mian”分支上运行分解推理时遇到以下问题：ValueError：没有足够的值可以解包（预期 4 个，但得到 2 个）。
实际的真实 kvcache 形状是“kv_cache[0] shape torch.Size([2162, 81920])”
INFO 12-03 14:31:48 model_runner_base.py:120] 将失败执行的输入写入 /tmp/err_execute_model_input_20241203-143148.pkl... INFO 12-03 14:31:48 model_runner_base.py:149] 已完成将失败执行的输入写入 /tmp/err_execute_model_input_20241203-143148.pkl。错误 12-03 14:31:48 engine.py:135] ValueError('模型执行错误（输入转储到 /tmp/err_execute_model_input_20241203-143148.pkl）：没有足够的值来解包（预期 4 个，得到 2 个）'）错误 12-03 14:31:48 engine.py:135] 回溯（最近一次调用最后一次）：错误 12-03 14:31:48 engine.py:135] 文件“/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner_base.py”，第 116 行，在 _wrapper 中错误12-03 14:31:48 engine.py:135] 返回 func(*args，**kwargs) 错误 12-03 14:31:48 engine.py:135] 文件“/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/worker/model_runner.py”，第 1718 行，在 execute_model 中错误 12-03 14:31:48 engine.py:135] get_kv_transfer_group().send_kv_caches_and_hidden_states(错误 12-03 14:31:48 engine.py:135] 文件“/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_transfer_agent.py”，第 60 行，在 send_kv_caches_and_hidden_states 中错误 12-03 14:31:48 engine.py:135] self.connector.send_kv_caches_and_hidden_states（错误 12-03 14:31:48 engine.py:135] 文件“/mnt/data_disk101/data_disk/lwq/LLM_INFER/split_platform/opensource/vllm-kuntai-disagg-refactor_1202/vllm/distributed/kv_transfer/kv_connector/simple_connector.py”，第 134 行，位于 send_kv_caches_and_hidden_states ERROR 12-03 14:31:48 engine.py:135] ，，num_heads，head_size = kv_cache[0].shape ERROR 12-03 14:31:48 engine.py:135] ValueError：没有足够的值来解压（预期 4 个，得到 2 个）ERROR 12-03 14:31:48 engine.py:135]
我的启动命令是：
CUDA_VISIBLE_DEVICES=3 nohup nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8100 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4 nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8200 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}' > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &
我的启动命令是：
CUDA_VISIBLE_DEVICES=3 nohup nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8100 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2}' > $Log_folder/p.log 2>&1 &
CUDA_VISIBLE_DEVICES=4 nohup python3 -m vllm.entrypoints.openai.api_server --model $model --port 8200 --max-model-len 1000 --gpu-memory-utilization 0.7 --kv-transfer-config '{"kv_connector":"PyNcclConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2}' > $Log_folder/D.log 2>&1 &
nohup python3 disagg_prefill_proxy_server.py > $Log_folder/proxy_server.log 2>&1 &

我猜你使用的是 Volta 或 Turing 架构的 GPU 卡？我在这个 PR 的旧版本中发现了这个问题。@KuntaiDu如果您没有带宽，我可以提出 PR 来解决这个问题。

NVIDIA A100-SXM4-80GB

NVIDIA A100-SXM4-80GB

OK, then this bug may affect a wider range than I thought. My solution is to obtain num_heads and head_size from model_executable.model.config instead of getting them from kv_cache[0].shape.

Hello, I noticed that in #6170  you used torch.distributed.init_process_group to initialize all ranks for prefill node and decode node, but later changed it to StatelessProcessGroup for kv cache transfer.
However, StatelessProcessGroup only supports nccl backend. If I want to use the CPU for transferring KV cache, do you have any good suggestions? It seems that TCPStore might not be suitable for transferring large amounts of data.

@liuyumoye can you take a look at #10884 ? I think mooncake transfer engine should support cpu transfer.

@liuyumoye can you take a look at #10884 ? I think mooncake transfer engine should support cpu transfer.

Thanks, I'll try your suggestion


A dropdown component

I've got a dropdown component which works similar to lookup.
It has also got default styles for items but not limited to them and can display random content.
I know this component is already being worked on.
Is it worth submitting a PR?

I'm interested in this component, as it's quite a common scenario and normally is quite all by itself. Would love to see it.
cc @EisenbergEffect

Live demo
https://gist.dumber.app/?gist=fbbfc92df36576847d6886dd051d1ceb

The component calculates the best dropdown position. Bottom-right is the preferred one, top and left are selected when dropdown dimensions don't fit AND there is more space on the other side of the trigger.

This looks nice. Is the positioning service insufficient to handle the requirement of this component?
cc @ben-girardet

As far as I understood, the service does not limit the dimensions of an element to fit in window. Am I right?

It's a component I would use but I question the name.  Maybe it's my window's background showing, but I always think of a dropdown component displaying a list of values and letting the user choose one.
This is more like a popup to me.

I'm not insisting on the name :)

As far as I understood, the service does not limit the dimensions of an element to fit in window. Am I right?

You're right @MaximBalaganskiy but the plan is to add this in the service. It would make lots of sense to use the positioning service for such a component.
I can help with that if necessary, only it will be in two weeks or so as I have some personal things going on right now. Let me know if I can help ;-)

I'll probably leave that service to you guys, don't feel comfortable modifying it.

What do we do with the dropdown/popup?

Let's add it in. For the service, maybe this component can get it first, and will join the list of components that use the positioning service. More motivation for @ben-girardet to enhance it.

I have a couple of questions around the naming of this component.   It's not my intention to bog down your work (which I'm sure this is).  I just want to easily explain to someone which component to use when.  This component is so flexible because of the use of  that it can be used for many scenarios.

Could this component be made to popup on the left, right, top, or bottom of a target?
Is there anything in this component that demands an item be chosen from the popup?
How is this component different than a menu component (and even the name 'menu' is too limiting IMO) ?  https://vuematerial.io/components/menu

Some other names I have seen:
Popover - https://material-ui.com/components/popover/
https://getbootstrap.com/docs/4.0/components/popovers/
And to really geek out: https://uxdesign.cc/pop-up-popover-or-popper-a-quick-look-into-ui-terms-cb4114fca2a
@EisenbergEffect do you want to weigh in on naming?

The component "prefers" bottom-left position. Switching to top and right only if there is not enough space to fit the content AND there is more space on the other side. Max width and height are set to fit into window.
The popup is closed on any click which reaches the window
The main difference is auto positioning, hence missing "directions" and "sizes" api. "Autoclose" is also missing.

Size can be controlled by setting the content size.
I'm, personally, fine with popup - it reflects the intent.


ruby 2.2. support in .travis.yml

AR 3.2 has issues with this version of Ruby. Please fix exclusion matrix accordingly.

@inossidabile ok


Will you support developing 1.18?

Minecraft is updated very quickly, many mods from 1.12 have been ported to 1.18, I hope this mod can be like them, so that you can watch TV in higher versions

Yes, it will be.


Use ConcurrentDictionary for static var

follow up https://github.com/tonerdo/coverlet/pull/383
I prefer use concurrent collection for a static in case of concurrent test/build
/cc @tonerdo @ViktorHofer

LGTM


"make tidy" fails to finish when trying to build operator-sdk from source ( build from master )

Bug Report
What did you do?
trying
$ git clone https://github.com/operator-framework/operator-sdk
$ cd operator-sdk
$ git checkout master
$ make tidy
$ make install
following instructions from
->  https://sdk.operatorframework.io/docs/installation/install-operator-sdk/#compile-and-install-from-master
What did you expect to see?
make tidy to finish
What did you see instead? Under which circumstances?
$ make tidy
go<EMAIL_ADDRESS>requires
<EMAIL_ADDRESS>unrecognized import path "vbom.ml/util": https fetch: Get "https://vbom.ml/util?go-get=1": dial tcp: lookup vbom.ml on <IP_ADDRESS>:53: no such host
make: *** [Makefile:83: tidy] Error 1

Environment
Fedora 32
$ go version
go version go1.14.7 linux/amd64
Operator type:
language go
Kubernetes cluster type:

$ operator-sdk version

$ go version (if language is Go)
go version go1.14.7 linux/amd64
Possible Solution
in this link
https://github.com/operator-framework/operator-sdk/issues/3859
it says that it will work with older go version, but Fedora 32 ships  go1.14.7  version and it would be good if building operator with it works too

seems this note Note: Ensure that your GOPROXY is set with its default value for Go versions 1.13+ which is https://proxy.golang.org,direct. from https://sdk.operatorframework.io/docs/installation/install-operator-sdk/#compile-and-install-from-master is necessary in order to make this working. Works after setting GOPROXY to above value. ... I think this can be closed.

Thank you @ekuric.
I am closing this one. Feel free to raise new issues if you need.


OneDrive 2.7 blocking legacy authentication is duplicative of AAD

💡 Summary
Should this setting be kept in the OneDrive Baseline and added to SharePoint or removed
Motivation and context
The user is already directed to use the setting for blocking legacy authentication in the AAD. When enabled for AAD, the blocking is applied for all Microsoft products including SharePoint & OneDrive. Should the setting be kept in OneDrive & added to SharePoint, but indicated to be selected if the are not implementing AAD baseline or as a backup? Or should this policy be removed from the OneDrive baseline.
Implementation notes
One or the other:

Policy added to Sharepoint
Policy removed from OneDrive

Acceptance criteria

[ ] Updated policy has the change noted in implementation notes
[ ] OneDrive or SharePoint Code is updated to have or not have the policy check

@schrolla Who would be the appropriate person to ask? Ted?

@tkol2022 need your exp

Please delete this from the OneDrive baseline. Blocking legacy authentication is the first policy in the AAD baseline and it covers all M365 products. It is one of the most important policies across all the products and is officially cited by both Microsoft and CISA (e.g. in their SolarWinds advisories) in guidance related to securing the M365 environment. Therefore, the AAD policy is a must implement for any organization serious about hardening M365.
A couple of related thoughts:

I am preparing a spreadsheet to perform a final adjudication of all the current Sharepoint and OneDrive policies, since there were numerous other items that may get removed. I will setup a meeting for this week to review them with you.
In general, items that are related to Authentication or Authorization, should be in the AAD baseline. The AAD baseline is not optional for an organization securing M365 - in fact it is probably the first one that should be implemented because it provides the foundational security of the identities and how they access the system.

Not commenting here based on my reading of the above--please flag anything you need CISA to decide once ready.

Closing as this was addressed in #370.


Use blurbs to determine set of edges

Pros: we will no longer need to store each character's adjacency list in chars.js, thus eliminating redundancy and making it easier to add content.
Cons: we will have to iterate through all (TAC, ASOIAF) pairs and ping the server for each one, so this may increase the time it takes the page to load.

I tried doing this using JSON XMLHttpRequest() but couldn't get it to work.

It appears that this is possible but will always result in a bunch of 404 errors being logged to the console.


Docker container somtimes is still running status.Although the proccess of container is killed

Docker container somtimes is still running status.
Although the proccess of container is killed, this
patch will fix it. see the issue:
https://github.com/docker/docker/issues/11809
Signed-off-by: Chao Chen<EMAIL_ADDRESS>

Ping @crosbymichael @LK4D4: WDYT?

@cc272309126 Sorry, but this is not solution. It is just conceal real problem, that sometime setStopped not called on container exit. If you have reproduction case, pls post and issue with it.
Thanks!


parser: read the functions' throws attribute, insert GError parameter

@EPashkin you might not like doing this in the parser, I'm open to alternatives. This seems like the only place that can easily see every function. Doing the same in the analysis modules will be more complicated.

No problem with this, it really parameter.
But may be better add as ParameterDirection::Error ?

Would replacing instance_parameter with kind: enum ParameterKind { Normal, Instance, Error } work?

Maybe. Not sure what we needed function generation.
So now merge as is, when we needed code with errors, then change.

No effect on current generated widgets.

@homu r+

:pushpin: Commit 8a2bc15 has been approved by gkoz

:hourglass: Testing commit 8a2bc15 with merge 657b802...

:sunny: Test successful - status


Rework CloudStack API comments

Summary
The CloudStack API comments didn't stick with our guidance of avoiding excessively long lines. This change carves up code comments into appropriate length lines and removes unnecessary comments.
See https://github.com/aws/eks-anywhere/blob/main/docs/developer/best-practice.md#line-length

/approve


CarlinKit disconnecting on it's own

Hello,
I picked up a Carlinkit CPC200-CCPA from Amazon.
When I plug it in, about 25 seconds later Removable medium is inserted window opens, and 10 seconds after that it disconnects.  When it disconnects a box in the top-right of the screen pops up saying:

Drive was removed without ejecting
Please use menu to eject before removal

If I run Carplay after it says it says that I disconnected it, I get the same errors as in #20. When I run lsusb the Carlinkit isn't listed.
If I run Carplay right after I plug the Carlinkit in, I get this error and then the app closes:

raspberry@raspberrypi:~/Desktop $ ./Carplay.AppImage
/home/raspberry/.config/ReactCarplay
spawning carplay {
dpi: 240,
nightMode: 0,
hand: 0,
boxName: 'nodePlay',
width: 799,
height: 479,
fps: 30
}
directory found
device not connected
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to open kms_swrast: /usr/lib/dri/kms_swrast_dri.so: cannot open shared object file: Permission denied (search paths /usr/lib/aarch64-linux-gnu/dri:$${ORIGIN}/dri:/usr/lib/dri)
failed to load driver: kms_swrast
MESA-LOADER: failed to open swrast: /usr/lib/dri/swrast_dri.so: cannot open shared object file: Permission denied (search paths /usr/lib/aarch64-linux-gnu/dri:$${ORIGIN}/dri:/usr/lib/dri)
failed to load swrast driver
carplay connection
carplay connection
sending dpi
FATAL ERROR:  LIBUSB_TRANSFER_ERROR
Aborted

When running lsusb the Carlinkit does show up until it disconnects itself.
Any advice?

Hello, since my last post, I got two new dongles. I got the same dongle as before from a different Amazon page to see if it was a problem with that specific dongle but it wasn't and I got the same behavior.
The other dongle I got was this, the wired Carplay one. This was linked to, from the YouTube video. When I run the code with this dongle I get this error:

/home/raspberry/.config/ReactCarplay
spawning carplay {
dpi: 240,
nightMode: 0,
hand: 0,
boxName: 'nodePlay',
width: 800,
height: 480,
fps: 30
}
Assets not present, downloading
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to retrieve device information
MESA-LOADER: failed to open kms_swrast: /usr/lib/dri/kms_swrast_dri.so: cannot open shared object file: Permission denied (search paths /usr/lib/arm-linux-gnueabihf/dri:$${ORIGIN}/dri:/usr/lib/dri)
failed to load driver: kms_swrast
MESA-LOADER: failed to open swrast: /usr/lib/dri/swrast_dri.so: cannot open shared object file: Permission denied (search paths /usr/lib/arm-linux-gnueabihf/dri:$${ORIGIN}/dri:/usr/lib/dri)
failed to load swrast driver
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100 8456k  100 8456k    0     0   614k      0  0:00:13  0:00:13 --:--:--  534k
file downloaded, unzipping
sending dpi
device connected and ready
FATAL ERROR:  LIBUSB_TRANSFER_ERROR
Aborted

Now the dongle doesn't disconnect randomly so small improvement. From what I can tell, the problem seems to be with the LIBUSB_TRANSFER_ERROR.
Any advice?

Hi sorry for the lack of reply!
take a look here for more info
https://forums.moderndaymods.com/t/carlinkit-is-not-connecting/87/10

Thanks, this works great on the wired Carplay dongle.
Is the .img for the newer firmware available for when the Node-Carplay update comes out? I'm afraid to update the wireless dongle since I don't want to be stuck with the old firmware forever.
Thanks for everything!


fix(apps/msgtst): Send multiple messages

Expand the messenger test to send multiple messages.

@BetsyRT r+

:pushpin: Commit 3c21f39 has been approved by handong32

:hourglass: Testing commit 3c21f39 with merge 301b19a...

:sunny: Test successful - runtests
Approved by: handong32
Pushing 301b19a48f335bcfd1ccc12647aa998fc18c4624 to master...


missing comma in documentation

Hi,
just wanted to report a minor issue.
https://pola-rs.github.io/polars-book/user-guide/coming_from_pandas.html
is missing a comma after col 'type'
df = pd.DataFrame({
"type": ["m", "n", "o", "m", "m", "n", "n"]
"c": [1, 1, 1, 2, 2, 2, 2],
})

Nice catch! You should submit a PR 😊

I can get this now if you want.

Btw @SnoozeFreddo you can add a code snippet by
```python
doing this
```

heh i keep doing that with gh cli. The merge targets my master. Let me fix.

Fixed :)

I just tried to make a pull request for another small issue in the docs text but i don't have the access rights to push. How can i make a pull request without permissions? @ritchie46 @braaannigan
There are also issues with the makefile

@SnoozeFreddo
It's the standard procedure for these projects:

you fork the repo
clone it to your machine
make a branch
make your changes
make a PR and check that it passes ci
See the Contributing.md for more details or ping a message on discord if you get stuck


build(deps): bump cheerio from 0.17.0 to 0.22.0

I meet some issues to migrate from 0.19.0 to 0.20.0
I think it's related to :root but I don't understand why it was used in this case

According to cheerio doc .root can be use to query the top level root element 👍


bpf_tests: Always RemoveMemlock()

Use RemoveMemlock() to avoid error seen on a local run of the bpf tests:
--- FAIL: TestBPF (0.02s)
bpf_test.go:149: new coll: map test_cilium_signals: map create: operation not permitted (MEMLOCK may be too low, consider rlimit.RemoveMemlock)
FAIL
Signed-off-by: Jarno Rajahalme<EMAIL_ADDRESS>

Seems like this is failing on Travis.

Also the codeowners for this file is missing so I added it: https://github.com/cilium/cilium/pull/22928


Can't process █

Lolcat is having trouble running for me when the unicode character 'FULL BLOCK' (U+2588) is in the input.
(I was fooling around with this script.)

Looks like lolcat doesn't work with any block character from this list.


Fix mbed compile --flash

Unknown error is thrown while executing mbed compile --flash:
[mbed] ERROR: Unknown Error: local variable 'baudrate' referenced before assignment
Add the required argument to compile_ method.
This also fixes another error while mbed compile --flash --baudrate 9600:
[mbed] ERROR: Unknown Error: compile_() got an unexpected keyword argument 'baudrate'
This is a regression introduced by https://github.com/ARMmbed/mbed-cli/pull/837

@theotherjimmy, @screamerbg: please consider this for 1.9.1: the regression in 1.9.0 was reported by various development teams in Cypress.

Thanks for reporting this. I'll address it shortly.

Thanks. 1.9.1 is out with this patch.


Improve ConsolidatorScanPriority comparison performance

Rationale
The ConsolidatorScanPriority type implements IComparable and is used as the priority in the SubscriptionManager's priority queue. Due to the way comparison is handled for the case when PriorityQueue is not supplied with IComparer instance explicitly, an intermediate ObjectComparer is used for the comparison that eventually delegates comparison to ConsolidatorScanPriority.CompareTo.
See the following screenshot (PerfView):

Potential Solution
The proposed solution is to introduce a new Comparer and supply it to the PrioorityQueue constructor. This yields 5-10 percent improvement in data points per second for all Lean benchmarks on my machine.
The PR is on the way.
Checklist

[x] I have completely filled out this template
[x] I have confirmed that this issue exists on the current master branch
[x] I have confirmed that this is not a duplicate issue by searching issues

@Martin-Molinero, I wanted to discuss something with you.
Looking at SubscriptionManager code, I see the potential for improvement in ScanPastConsolidators by replacing PriorityQueue with plain List. The main question here is whether it's critical that we have the order of consolidator scanning dictated by the Priority?
Roughly, I'm thinking about something like this:
public void ScanPastConsolidators(DateTime newUtcTime, IAlgorithm algorithm)
{
    if (_consolidatorsToAdd != null)
    {
        lock (_threadSafeCollectionLock)
        {
            _consolidatorsToAdd.DoForEach(x => _consolidatorsSortedByScanTime.Add(x.Item1));
            _consolidatorsToAdd = null;
        }
    }

    for (int i = 0; i < _consolidatorsSortedByScanTime.Count; I++)
    {
        var consolidatorToScan = _consolidatorsSortedByScanTime[I];
        var priority = consolidatorToScan.Priority;

        if (priority.UtcScanTime >= newUtcTime)
        {
            continue;
        }

        if (consolidatorToScan.Disposed)
        {
            // consolidator has been removed
            continue;
        }

        if (priority.UtcScanTime != algorithm.UtcTime)
        {
            // only update the algorithm time once, it's not cheap because of TZ conversions
            algorithm.SetDateTime(priority.UtcScanTime);
        }

        if (consolidatorToScan.UtcScanTime <= priority.UtcScanTime)
        {
            // only scan if we still need to
            consolidatorToScan.Scan();
        }
    }
}

This change shows drastic improvement in execution time at the expense of sacrificing the exact scan order. From my perspective, we might be fine with that, but I'm not sure.
What do you think?

Hey @starteleport! yes sadly we need 100% determinism, original impl was actually a list I think and we had to add priority to make sure it would trigger every time the in the same way, and it needs to scale to thousands of entries (imagine hundreds of symbols each with a few indicators/consolidators), that pushed us to the priority queue

@Martin-Molinero could you please help me grasp the need for taking UtcScanTime into account when determining order?
I did some research myself, there were two issues and their corresponding pull requests that contributed to the current implementation:

https://github.com/QuantConnect/Lean/issues/5457
https://github.com/QuantConnect/Lean/issues/8017

The second one is about introducing consolidator ID to guarantee ordering between consolidators updating at the same time. The rationale is clear to me.
The first issue was devoted to missing consolidation in no data situations. From the issue and the comments it's not obvious to me why UtcScanTime is being used in comparison. I can see that we use UtcScanTime inside the condition of the while loop to skip those consolidators that we don't need at the current time step. But is it the only reason?
Now bear with me, the reason why I keep asking is that the time complexity of ScanPastConsolidators is O(n log n), where n is the number of consolidators. Reason being, in the worst case we need to go through all elements of the queue (while) and do Enqueue and Dequeue on each step which is O(log n) each (https://stackoverflow.com/a/74181115/1845402).
If UtcScanTime was only included in sorting to limit the number of elements considered, and nothing more, then we could improve the performance in this spot.
Assuming it is the case, we could get away with SortedList which sorts by consolidator ID only. That way, the list would ever change (O(n)) if/when consolidators are added/removed. Other times, we'll just need to do a O(n) loop through the list, skipping items that we don't need to scan yet. Given that thousands of consolidators are expected, I think it's worth considering at least.
@jaredbroad, tagging you as well as the reviewer of the related PRs
To sum up, I would really love to learn more about why's of this code, and even if my current proposal is not feasible, I might be able to address it another way.
Just to give an idea: switching to plain List (I haven't tried with SortedList yet, but I don't expect much difference) yielded more than 2x improvement in IndicatorRibbonBenchmark with 50 consolidators.


Switch to cobra/pflag for CLI

The code in this PR switches the CLI library from go-flags to cobra. This prepares us for a config file.

Current coverage is 51.62% (diff: 45.58%)

Merging #631 into master will decrease coverage by 0.15%

@@             master       #631   diff @@
==========================================
  Files            91         93     +2
  Lines          7469       7437    -32
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           3867       3839    -28
- Misses         3026       3043    +17
+ Partials        576        555    -21

Powered by Codecov. Last update eeec0d6...3601c39


Schedule endpoints error

Any call to the new Schedule endpoints returns:
Error unmarshalling body: json: cannot unmarshal object into Go struct field APIResponse.data of type []interface {}
schedule icalendar endpoint returns:
Error unmarshalling body: invalid character ‘B’ looking for beginning of value.
Similar errors across all the /schedule endpoints.
Error was present and consistent in 1.0 and 1.0.2 builds.

Is this for all endpoints? If so- I believe there's actually a fix already in the nightlies that resolves this; if not, please let me know a sample call and happy to dig further.

I haven't tested Create, Update, or Delete via CLI.  I'm unable to pull the ID of the segment from the GET to test.
I'm still receiving the error from:
twitch api get schedule -q broadcaster_id=foo
Error unmarshalling body: json: cannot unmarshal object into Go struct field APIResponse.data of type []interface {}
Even with the -u unformatted flag.
Might be able to work around test data this weekend to attempt the Create, Update, and Delete operations.

Got it. This seems to be tied to this specific version, but changes in the next release (which is being cut very shortly) will resolve this. I'll keep this open for now, but will close out with the next release.


Ansible failed

I tried the install, and it failed on this particular error.
TASK[homebrew: homebrew | bundle] ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** **
	fatal: [localhost]: FAILED! => {
		"changed": true,
		"cmd": ["brew", "bundle"],
		"delta": "0:00:18.502421",
		"end": "2017-09-21 22:12:43.861571",
		"failed": true,
		"msg": "non-zero return code",
		"rc": 1,
		"start": "2017-09-21 22:12:25.359150",
		"stderr": "Warning: Use neovim instead of deprecated neovim/neovim/neovim\nError: No available formula with the name \"srm\" ",
		"stderr_lines": ["Warning: Use neovim instead of deprecated neovim/neovim/neovim", "Error: No available formula with the name \"srm\" "],
		"stdout": "Using caskroom/cask\nUsing caskroom/versions\nUsing homebrew/bundle\nUsing homebrew/core\nUsing homebrew/dupes\nUsing homebrew/services\nUsing homebrew/versions\nUsing neovim/neovim\nUsing abook\nUsing ack\nUsing ant\nUsing aspcud\nUsing autoconf\nUsing automake\nUsing awscli\nUsing bdw-gc\nUsing berkeley-db\nUsing cabal-install\nUsing cairo\nUsing camlp4\nUsing chruby\nUsing clasp\nUsing clipper\nUsing cloc\nUsing cloog\nUsing closure-compiler\nUsing cmake\nUsing coreutils\nUsing cscope\nUsing ctags\nUsing dirmngr\nUsing elinks\nUsing emacs\nUsing ffmpeg\nUsing fontconfig\nUsing freetype\nUsing fswatch\nUsing fzf\nUsing gcc\nUsing gd\nUsing gdb\nUsing gdbm\nUsing gdk-pixbuf\nUsing gettext\nUsing ghc\nUsing gifsicle\nUsing git\nUsing glib\nUsing gmime\nUsing gmp\nUsing gnu-sed\nUsing gnupg\nUsing gnupg2\nUsing gnutls\nUsing go\nUsing gobject-introspection\nUsing gpg-agent\nUsing gringo\nUsing harfbuzz\nUsing highlight\nUsing htop-osx\nUsing httping\nUsing httrack\nUsing hub\nUsing icu4c\nUsing ievms\nUsing iftop\nUsing imagemagick\nUsing imapfilter\nUsing isl\nUsing isync\nUsing jemalloc\nUsing jpeg\nUsing jq\nUsing json-c\nUsing lame\nUsing less\nUsing lbdb\nUsing libassuan\nUsing libcroco\nUsing libevent\nUsing libffi\nUsing libgcrypt\nUsing libgpg-error\nUsing libksba\nUsing libmpc\nUsing libpng\nUsing librsvg\nUsing libtasn1\nUsing libtermkey\nUsing libtiff\nUsing libtool\nUsing libusb\nUsing libusb-compat\nUsing libuv\nUsing libvo-aacenc\nUsing libvterm\nUsing libyaml\nUsing libyubikey\nUsing links\nUsing lockrun\nUsing lua\nUsing macvim\nUsing makedepend\nUsing mariadb\nUsing memcached\nUsing mldonkey\nUsing mobile-shell\nUsing mpfr\nUsing msgpack\nUsing msmtp\nUsing mutt\nUsing neovim/neovim/neovim\nUsing nettle\nUsing nginx\nUsing node\nUsing notmuch\nUsing ocaml\nUsing ocamlbuild\nUsing openssl\nUsing<EMAIL_ADDRESS>optipng\nUsing ossp-uuid\nUsing pandoc\nUsing pango\nUsing par\nUsing pcre\nUsing perl\nUsing pinentry\nUsing pixman\nUsing pkg-config\nUsing pngquant\nUsing postgresql\nUsing potrace\nUsing protobuf\nUsing pstree\nUsing pth\nUsing pwgen\nUsing python\nUsing python3\nUsing qemu\nUsing qt\nUsing ragel\nUsing ranger\nUsing readline\nUsing reattach-to-user-namespace\nUsing redis\nUsing rhino\nUsing ripgrep\nUsing ruby-install\nUsing s3cmd\nUsing shared-mime-info\nUsing shellcheck\nUsing sqlite",
		"stdout_lines": ["Using caskroom/cask", "Using caskroom/versions", "Using homebrew/bundle", "Using homebrew/core", "Using homebrew/dupes", "Using homebrew/services", "Using homebrew/versions", "Using neovim/neovim", "Using abook", "Using ack", "Using ant", "Using aspcud", "Using autoconf", "Using automake", "Using awscli", "Using bdw-gc", "Using berkeley-db", "Using cabal-install", "Using cairo", "Using camlp4", "Using chruby", "Using clasp", "Using clipper", "Using cloc", "Using cloog", "Using closure-compiler", "Using cmake", "Using coreutils", "Using cscope", "Using ctags", "Using dirmngr", "Using elinks", "Using emacs", "Using ffmpeg", "Using fontconfig", "Using freetype", "Using fswatch", "Using fzf", "Using gcc", "Using gd", "Using gdb", "Using gdbm", "Using gdk-pixbuf", "Using gettext", "Using ghc", "Using gifsicle", "Using git", "Using glib", "Using gmime", "Using gmp", "Using gnu-sed", "Using gnupg", "Using gnupg2", "Using gnutls", "Using go", "Using gobject-introspection", "Using gpg-agent", "Using gringo", "Using harfbuzz", "Using highlight", "Using htop-osx", "Using httping", "Using httrack", "Using hub", "Using icu4c", "Using ievms", "Using iftop", "Using imagemagick", "Using imapfilter", "Using isl", "Using isync", "Using jemalloc", "Using jpeg", "Using jq", "Using json-c", "Using lame", "Using less", "Using lbdb", "Using libassuan", "Using libcroco", "Using libevent", "Using libffi", "Using libgcrypt", "Using libgpg-error", "Using libksba", "Using libmpc", "Using libpng", "Using librsvg", "Using libtasn1", "Using libtermkey", "Using libtiff", "Using libtool", "Using libusb", "Using libusb-compat", "Using libuv", "Using libvo-aacenc", "Using libvterm", "Using libyaml", "Using libyubikey", "Using links", "Using lockrun", "Using lua", "Using macvim", "Using makedepend", "Using mariadb", "Using memcached", "Using mldonkey", "Using mobile-shell", "Using mpfr", "Using msgpack", "Using msmtp", "Using mutt", "Using neovim/neovim/neovim", "Using nettle", "Using nginx", "Using node", "Using notmuch", "Using ocaml", "Using ocamlbuild", "Using openssl", "Using<EMAIL_ADDRESS>"Using optipng", "Using ossp-uuid", "Using pandoc", "Using pango", "Using par", "Using pcre", "Using perl", "Using pinentry", "Using pixman", "Using pkg-config", "Using pngquant", "Using postgresql", "Using potrace", "Using protobuf", "Using pstree", "Using pth", "Using pwgen", "Using python", "Using python3", "Using qemu", "Using qt", "Using ragel", "Using ranger", "Using readline", "Using reattach-to-user-namespace", "Using redis", "Using rhino", "Using ripgrep", "Using ruby-install", "Using s3cmd", "Using shared-mime-info", "Using shellcheck", "Using sqlite"]
	}
to retry, use: --limit @ / Users / priyankkapadia / .ansible / darwin.retry

PLAY RECAP ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** ** *
	localhost: ok = 18 changed = 1 unreachable = 0 failed = 1

Any ideas on this ?
After this i tried ./install again and it failed with "too many symlinks: bad interpreter error"

"stderr_lines": ["Warning: Use neovim instead of deprecated neovim/neovim/neovim", "Error: No available formula with the name \"srm\" "],

Looks like there is no "srm" formula anymore. I guess it's either been removed or renamed. Will need to update the Brewfile.

After this i tried ./install again and it failed with "too many symlinks: bad interpreter error"

Not sure what that is about, but one thing to note: install does a lot of stuff (see the roles folder) and I wouldn't expect that you'd want to do all of it anyway;  see ./install --help for how to run just a subset.

Dropped "srm" in 365115f, and silenced the "neovim" warning in 36816f9.

is there a way to revert the changes ?

i am sorry for being ignorant.. this is one of my screw ups :(

is there a way to revert the changes ?

Depends which changes you are talking about. Uninstalling an unwanted Homebrew-installed package, for example, is usually a matter of brew uninstall [the-package-name]. Things like dotfiles, before overwriting a pre-existing file with a symlink, backups get made in ~/.backups.

Going to close this as I've addressed those two concrete issues in the commit I linked to above. We can deal with any other issues separately.