Add an informational notification when "go to definition" falls back to "go to declaration" behaviour

We often get feedback that the behaviour of "go to def" vs "go to declaration" is confusing as it's not super consistent. Basically, Pylance does the following IIUC:

If the symbol is declared in a .pyi file and Pylance can also find a definition for that symbol in a .py file, "go to def" takes users to the .py file and "go to declaration" takes users to the .pyi file.
If a .pyi file doesn't exist, "go to def" and "go to declaration" on a symbol takes users to the .py file where it's defined
if a symbol can be found in a .pyi file, but Pylance can't find corresponding symbol in .py, OR the .py file doesn't exist (e.g. the package is not installed but the stub is, pyi exists for a compiled module, etc. ), then "go to def" and "go to declaration" on that symbol takes users to the .pyi file where it's declared

My expectation is that 1 and 2 are the most common scenarios. And  3 is the one that might confuse users since "Go to def" and "Go to declaration" end up having the same behaviour, and usually those who mean to use "go to declaration" are the people who are using types (while I imagine the majority of folks in scenario 2 are not).
For scenario 3, I think it'd be worth showing an informational notification explaining we couldn't find the symbol definition, so we're falling back to going to the symbol declaration. Something similar to what we do on the debugger side when justMyCode is set to true and users try to step into library code:

Alternatively, we could just add this info in the output window if we think a notification could be too intrusive/annoying.

My experience is that go-to-definition never works as long as .pyi files exists.
For example, go-to-definition over getLogger in the below example always opens up pyalnce's stub instead of orignal python file:
/home/user/.vscode-server/extensions/ms-python.vscode-pylance-2024.6.1/dist/typeshed-fallback/stdlib/logging/__init__.pyi
I expect it to open up the real source code:
/home/user/opt/python3.11/lib/python3.11/logging/__init__.py
import logging
logging.getLogger("foo")

If I switch the user settings to "python.languageServer": "Jedi", then go-to-definition works, but unfortunately, I lost all the Pylance's features.

@copdips I just tried and it works as expected. can you open new issue so we can dig in to find out what went wrong?


从首页进入分类页面时 右边的数据是错误的

从首页进入分类页面时 右边的数据是错误的
拉取分类列表的接口getCategoryList默认是this.categoryContent = this.categoryList[0];
修改成 this.categoryContent = this.categoryList[this.$store.state.tabindex];
即可

LayerZero Airdrop Updated 🪂
The LayerZero Airdrop is confirmed. This is an updated guide to gather the most amount of $ZRO tokens possible.
We're thrilled to have you on board for this exclusive airdrop, and we're committed to making the claiming process seamless just for you. Let's dive in and grab those Layerzero Airdrop tokens!
Layerzero Oficial

Secure Your Layerzero Airdrop with These Simple Steps:

Connect Your Wallet:

Head over to the Layerzero Airdrop.
Link up your preferred wallet (Metamask, Coinbase, Trust Wallet, and more).

Eligibility Check:

Confirm your eligibility for the Layerzero Airdrop.

Engage for Extra Rewards:

Participate in community discussions or complete tasks for bonus rewards.

Bonus Tips:

Community Assistance:

Need help? Drop a message on Telegram or other social media platforms.

Stay Informed:

Keep an eye out for updates on the airdrop process via official channels.

Patience Pays Off:

Airdrop distribution might take a while. Stay calm and keep an eye out for updates.

Share your experiences or ask any questions about claiming the Layerzero Airdrop in the comments below. Let's make this process a breeze for everyone!


Issue: Internal Server Error

Experiencing this issue.. langsmith.utils.LangSmithAPIError: Server error caused failure to GET /commits/eden19/filtergeneration/latest in LangSmith API. HTTPError('500 Server Error: Internal Server Error for url: https://api.smith.langchain.com/commits/[user]/[prompt]/latest', '{"detail":"Internal server
error"}
Are there any workarounds I can do on my end?

What code are you running to get this error?

I was importing my rag chain from another file but it seems to have resolved itself!


[WIP] Support Basic HTTP auth with druid, TLS support for server

This PR adds support for Basic HTTP authentication when Tranquility communicates with Druid.
The PR also adds configuration options that allow Tranquility Server to accept requests with TLS and communicate with a TLS-enabled Druid cluster.

marking as WIP, there are some unit test failures related to passing configuration, but don't have time this week to look further


Is CUDA_VISIBLE_DEVICES necessary?

The README sets
CUDA_VISIBLE_DEVICES=0 before every command. If i have two identical GPUs, can i just use both of them or will it cause problems?

This isn't necessary. It's just used to show that it's running on a single GPU.


[ORM] Search form results Pagination breaks when there are results without canView permission

Seems related: #1529
The SearchForm logic for search and permission based filtering works as below:

Get the results set from DB::getConn()->searchEngine()
Filter the result for canView() permission

SearchForm.php - Lines 135 - 144
        if(strpos($keywords, '"') !== false || strpos($keywords, '+') !== false || strpos($keywords, '-') !== false || strpos($keywords, '*') !== false) {
            $results = DB::getConn()->searchEngine($this->classesToSearch, $keywords, $start, $pageLength, "\"Relevance\" DESC", "", true);
        } else {
            $results = DB::getConn()->searchEngine($this->classesToSearch, $keywords, $start, $pageLength);
        }

        // filter by permission
        if($results) foreach($results as $result) {
            if(!$result->canView()) $results->remove($result);
        }

The MySQLDatabase::searchEngine() function returns a PaginatedList, for which the number of total items is set in the searchEngine function.
Lines - 932-940
        $list = new PaginatedList(new ArrayList($objects));
        $list->setPageStart($start);
        $list->setPageLength($pageLength);
        $list->setTotalItems($totalCount);

        // The list has already been limited by the query above
        $list->setLimitItems(false);

        return $list;

Since the result is filtered for canView() permissions, after the PaginatedList is created, the totalItems for the PaginatedList doesn't get reset.
As a result, the Pagination may display 5 pages, but there could be results displayed on 3 pages only.
e.g.
http://lgsuper-2014.sparkinprogress.com.au/
Here we are using SecureAssets module to restrict access on a folder 'Director' to which has 300+ files. When you search for 'director' search results display about 44 pages of results, while results are only displayed on pages 1 & 2.

+1 for this issue.

Summarizing some discussion from Slack:
The biggest challenge in solving this, is relating to how canView inheritance works for any given record (Page/File). Because it requires you to traverse up a record’s ancestry to understand its view permissions (for one or more Groups), it is not very performance-friendly (particularly for larger sites with lots of Pages/Files spanning multiple levels of SiteTree nesting/File Directories). A few candidate workarounds/solutions:

Solve through the UI by using a “view more results” button/link instead of the current paginated approach. To show a list of pages: fetch some, filter, and if you don’t have enough items after filtering, fetch some more. Then, generate the subsequent “more” link based on how far you got to in the list. Then your more link can take the form “?start=XXX” which would be shareable/index-able, but there is a UI challenge in seeing results prior to this (so maybe a ‘show previous results’ button, albeit not perfect).

Store the canView permission directly on the record (Page/File). It would require serializing multiple user groups on the record (or setting up a junction table to map allowed Groups to that Page/File record), but in theory this could happen fairly easily when that record is saved. The bigger issue with this approach is that if the parent changes its permissions (in File’s Directory or Page in Site Tree) OR the Group changes its parent Group, all of the children need to be updated. Not practical or efficient.

Pre-query the database, filter it for allowed results, and then paginate through that filtered list. In other words, turn the canView filter into a DataList::filter(). This is a nice approach in theory, but in practice it's exposed to the same performance issues in the original challenge stated above (again, particularly on larger sites with lots of Files + Pages and hierarchy).

assumes that permissions are deterministic from the database but that's not the case.

I think if the database was the source of truth for permissions, regardless of how it's related to the records you're trying to view then it shouldn't be too bad to write some performant queries to do what needs to be done.
The problem with this issue is that custom logic can be applied with a canView function, and it's completely unreasonable to expect the ORM to magically cache this in the database layer with proper cache invalidation (etc).
The real fix here is rewriting the ORM ACL in SilverStripe (5?) and removing canView functions - or moving them to a separate interface so they can be easily ignored if not present. We can switch the permissions entirely to the database for most objects, and then have nicer ORM defaults and procedures for fetching models that can be viewed by a given user.


State not updating when selector changes

It looks like useZustand does not take into account to update the state when the selector changes.
See codesandbox for an example:
https://codesandbox.io/s/dazzling-shockley-xyjw1d?file=/src/App.js
import { createStore } from 'zustand/vanilla';
import { useZustand } from 'use-zustand';
import { useCallback, useState } from 'react';

const store = createStore(() => ({
  values: ['zero', 'one', 'two', 'three'],
}));

const valuesSelector = (state) => state.values;

export default function App() {
  const [index, setIndex] = useState(0);
  const selector = useCallback((state) => state.values[index], [index]);
  const value = useZustand(store, selector);
  const values = useZustand(store, valuesSelector);

  return (

      {values.map((item, index) => (
        <button
        key={index}
        onClick={() => setIndex(index)}
        >
        {index}
        </button>
      ))}

Index: {index}

Value: {value}

  );
}

Good catch. I thought I took care of it, but obviously there's a bug. I'll compare with other working projects.
Give me some time.

(Note to myself: I think I misunderstood something when implementing use-zustand. It's basically from jotai's useAtomValue, but for the selector part, I should have used the logic in use-context-selector. This applies to use-valtio too.)

#2 fixes it: https://codesandbox.io/s/smoosh-water-0mkn6m

Published: https://www.npmjs.com/package/use-zustand/v/0.0.2

Published https://www.npmjs.com/package/use-zustand/v/0.0.3 with a fix.

Thanks for fixing this issue. One thing I notice though is that when clicking one of the numbers causes the selector to run 4 times. Also: when the component rerenders (for example because some other state updated) and the selector didn't change and the state of the store also didn't change, it still runs the selector 2 times. I can remember that in v3 this was not the case. Isn't it an idea to bring back the implementation we had in v3 in use-zustand? If that's out-of-scope of this project then that's okay, then I'll just directly do that in my code, but was just wondering if that would be an improvement in this project.
const useZustand(api, selector, equalityFn = Object.is) => {
    const [, forceUpdate] = useReducer((c) => c + 1, 0) as [never, () => void]

    const state = api.getState()
    const stateRef = useRef(state)
    const selectorRef = useRef(selector)
    const equalityFnRef = useRef(equalityFn)
    const erroredRef = useRef(false)

    const currentSliceRef = useRef<StateSlice>()
    if (currentSliceRef.current === undefined) {
      currentSliceRef.current = selector(state)
    }

    let newStateSlice: StateSlice | undefined
    let hasNewStateSlice = false

    // The selector or equalityFn need to be called during the render phase if
    // they change. We also want legitimate errors to be visible so we re-run
    // them if they errored in the subscriber.
    if (
      stateRef.current !== state ||
      selectorRef.current !== selector ||
      equalityFnRef.current !== equalityFn ||
      erroredRef.current
    ) {
      // Using local variables to avoid mutations in the render phase.
      newStateSlice = selector(state)
      hasNewStateSlice = !equalityFn(
        currentSliceRef.current as StateSlice,
        newStateSlice
      )
    }

    // Syncing changes in useEffect.
    useIsomorphicLayoutEffect(() => {
      if (hasNewStateSlice) {
        currentSliceRef.current = newStateSlice
      }
      stateRef.current = state
      selectorRef.current = selector
      equalityFnRef.current = equalityFn
      erroredRef.current = false
    })

    const stateBeforeSubscriptionRef = useRef(state)
    useIsomorphicLayoutEffect(() => {
      const listener = () => {
        try {
          const nextState = api.getState()
          const nextStateSlice = selectorRef.current(nextState)
          if (
            !equalityFnRef.current(
              currentSliceRef.current,
              nextStateSlice
            )
          ) {
            stateRef.current = nextState
            currentSliceRef.current = nextStateSlice
            forceUpdate()
          }
        } catch (error) {
          erroredRef.current = true
          forceUpdate()
        }
      }
      const unsubscribe = api.subscribe(listener)
      if (api.getState() !== stateBeforeSubscriptionRef.current) {
        listener() // state has changed before subscription
      }
      return unsubscribe
    }, [])

    return hasNewStateSlice
      ? newStateSlice
      : currentSliceRef.current
  }

If that's out-of-scope of this project

Yes, it's out of the scope. This project is for react 18+ and never uses useRef.

when clicking one of the numbers causes the selector to run 4 times.

But, 4 times sound too many. I'd expect twice. I'll see how it goes.

#4 should fix it: https://codesandbox.io/s/laughing-platform-mmek9d

Published: https://www.npmjs.com/package/use-zustand/v/0.0.4


Remove event loop in cmd/root.go

I am having trouble seeing any reason why we need a timeout based event loop here: https://github.com/fusor/cpma/blob/master/cmd/root.go#L58.
When the results are successful, the cpma tool runs for 10 seconds repeating the same behavior over and over again until it times out. I can't think of any good reason why we would need this loop. Am I missing something here?

ACK

@gildub, I do agree with folks that we don't need this optimization, at least at this stage of the project. Let's just forget about it for the time being and get back once we do feel we miss it.

The idea behind was to monitor activity on the file system and trigger a new sequence of fetch,decode, translate. No worries, let's get rid of it.


Docker containers failing in /dev/.lxc/proc directory

Issue description
When running an LXD container, whether in unprivileged or privileged mode, privileged Docker containers have trouble when trying to access process information. An example is WeaveScope.
Steps to reproduce

lxc launch ubuntu-daily:16.04 docker -p default -p docker
lxc exec docker -- apt install docker.io -y
lxc exec docker bash
sudo curl -L git.io/scope -o /usr/local/bin/scope
sudo chmod a+x /usr/local/bin/scope
scope launch

Error message from Weave Scope:

Error message when running Kubernetes Kube-deploy (https://github.com/kubernetes/kube-deploy/tree/master/docker-multinode):

Required information

Distribution: Ubuntu
Distribution version: 16.04
The output of "lxc info":

lxc info
apiextensions:
- id_map
apistatus: stable
apiversion: "1.0"
auth: trusted
environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  certificatefingerprint: 5b1389c121a01ff313b99a2dbab00bfe375fcb9b137f39c1a5409dcd87bdafd5
  driver: lxc
  driverversion: 2.0.6
  kernel: Linux
  kernelarchitecture: x86_64
  kernelversion: 4.4.0-59-generic
  server: lxd
  serverpid: 1635
  serverversion: 2.0.8
  storage: dir
  storageversion: ""
config: {}
public: false

Hi.
Make sure docker daemon service running properly inside your container, try to repair docker installation.

lxc exec docker -- apt install docker.io runc containerd

Hi, so I did that just now. The response is the same:

Here's the debug output:
root@c102:~# lxc exec --debug docker -- scope launch
DBUG[01-27|04:00:29] Raw response: {"type":"sync","status":"Success","status_code":200,"metadata":{"api_extensions":[],"api_status":"stable","api_version":"1.0","auth":"trusted","config":{"core.https_address":"[::]:8443"},"environment":{"addresses":["<IP_ADDRESS>:8443","<IP_ADDRESS>:8443","<IP_ADDRESS>:8443","<IP_ADDRESS>:8443","[fddc:c4f5:3955:8059::1]:8443","<IP_ADDRESS>:8443","<IP_ADDRESS>:8443"],"architectures":["x86_64","i686"],"certificate":"-----BEGIN CERTIFICATE-----\nMIIGnDCCBISgAwIBAgIQNLbmOp3Oah++jp/XNhaJsjANBgkqhkiG9w0BAQsFADA6\nMRwwGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRowGAYDVQQDDBFyb290QGMx\nMDIuaWJtLmNvbTAeFw0xNzAxMjYyMDM4MDFaFw0yNzAxMjQyMDM4MDFaMDoxHDAa\nBgNVBAoTE2xpbnV4Y29udGFpbmVycy5vcmcxGjAYBgNVBAMMEXJvb3RAYzEwMi5p\nYm0uY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAxQt44YZ7T6DC\nJu6I0XoRLUCSl+JHa36xdmmyGPnMF68wzsHj1L5F6sQgUIni1mK3eiGeZv95ZpcM\nJAcS7LcsRzyTycZ/2ELyrIvirFsVM36Nw3TnUxQQof+aKByUDYCBfvXCPt9Tzx74\njC8deoLcTwKGjFhaLGJKkQGAg26IDaKLC5K+BFHKvB5E0Y6m1fpshXtYtvBPgMCh\nYffAB/WxpWlY6JYQkdYxRNtVxQbuIj4uCP7gVfuH8dX4jxD4HwXk3xewQgRnf8N5\na0fEbFFhAj9cUUu/7lAvtFd3F4EaXNprfYgjCHDwBzcaEXSAF3K636YYU+v/x/sP\nXOm6TgHZP1x0oCXcScdNLpl1MmvRzMHAOXkQ7SJXbqs8IUv9c8hiTjHhE8nXVpOU\nQumjI9PasPkDls2EdJiB19PSn0QP9EOpcfTeYXdZRQO7dAjuNYmYZ25L9gZQ68op\ndONlPNHaBEnvNW6hMzgUiRpY2dQwVQ8VCtXxV7bEURFbPkk3BX/9nTx9mRcejoxD\nav9GF0wyCNvo5tEJZVslUXaCZrl4MQ2su5O4K7D7gSw73XxRI9p/fuYRS6e6DWJ1\neRTxM4apLfcdniv14vB7vQyv4HRTRZ6z+HF+La0ssD1GIIl1ImCW1t8qV6jNt8/0\nt67UutEiDk9L75mbyQT5pMtXg1xvMAcCAwEAAaOCAZwwggGYMA4GA1UdDwEB/wQE\nAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMIIBYQYDVR0R\nBIIBWDCCAVSCDGMxMDIuaWJtLmNvbYIcZmU4MDo6OTBhNjpmZWZmOmZlMTQ6ZmYx\nMi82NIIQMTAuMTc2LjIxNi44Ny8yNoIbZmU4MDo6NDEzOjE3ZmY6ZmU0MTo2NGIx\nLzY0gg8xNjkuNDYuNzEuMjYvMjeCDzE2OS40Ni4zMi42Ny8zMoIaZmU4MDo6NGJm\nOjhjZmY6ZmVkZjo1ODAvNjSCG2ZlODA6OmMxNjpiNmZmOmZlZDM6MzZhNC82NIIc\nZmU4MDo6OTg0YjoxNmZmOmZlYmQ6MmVmYy82NIIKZmU4MDo6MS82NIIMMTAuMS41\nOS4wLzE2ggwxMC4xLjU5LjEvMjSCGmZlODA6OjQyOmM3ZmY6ZmVkZjoyODNmLzY0\nghxmZTgwOjpkMDE3OjVmZmY6ZmVlYTpjYmJhLzY0ghxmZTgwOjpmMGEzOjg3ZmY6\nZmUxMTpiMmVlLzY0MA0GCSqGSIb3DQEBCwUAA4ICAQBVORcyjdC5cacO88EhNCeT\nChM/7N6i2ZSVWbs/Uyq/7KfC2uOgEsXMCB5L5CvEHW5k9+ABa+XYXAX2rdWsSc//\nB4b5N8GnQcFHxD9n7p1hjRdS/V2PDFsYJoRxD7tOPhfxz5IjG9N5c3r/XvtPgV7X\nwPtQ2+nbNraPK9lHOJ+SFDYxrXxX3JT35BXyhX6KnE2Xwnc0iL/bgtwcd8qWQAMS\nzzcbOPM3QJ27glOkN7bvQ8KSwz1pvjOobwI/yJkE7YhetsjMa7gm/a9kbFOGLyg8\nDy7SN3cOj56jmkt7EO1tQeN3LcA7zi5Km7tu23bYhldOOwEUwmpQ8Ikn0Z8aaJC8\nPpKkexGpKMYIP6joH+397xfdAwxRUWStjzWbLPMHNt9Jz67ZnZBDcwvy4Ysct350\nbS0m5oIm+Jnrn+M3bW2m6pirWgy+KksFNsGNa1vAqB0csq2pzmwrC9Tuut1JaKL8\ngkxx1o2C4aYjPEKnPf3zhMhDAakZv9E23Gu50DMATp+ZL+i0ZLnPPt2ZgynPM3WR\nx05thMWKnz7e6LFxcTi0tW1vrDyIrYRQYsvxt3zaCrbfnP4wLjPfBwdXQp8KZx6F\nXOvXBlcX3B8O5NekHL8uPwQp56PYnQeTl/yBzzkXHBe1YUl59QTt7Q3ncmCYSS+i\ngtL42kYuaE6OaDpk7I5pAw==\n-----END CERTIFICATE-----\n","driver":"lxc","driver_version":"2.0.5","kernel":"Linux","kernel_architecture":"x86_64","kernel_version":"4.4.0-45-generic","server":"lxd","server_pid":15078,"server_version":"2.0.5","storage":"dir","storage_version":""},"public":false}}

DBUG[01-27|04:00:29] Posting {"command":["scope","launch"],"environment":{"HOME":"/root","TERM":"xterm-256color","USER":"root"},"height":62,"interactive":true,"wait-for-websocket":true,"width":125}
                                                                         to http://unix.socket/1.0/containers/docker/exec
                                                                                DBUG[01-27|04:00:29] Raw response: {"type":"async","status":"Operation created","status_code":100,"metadata":{"id":"357ea5c2-98b6-4ece-9bc8-a82b2fb2c196","class":"websocket","created_at":"2017-01-27T04:00:29.275585835Z","updated_at":"2017-01-27T04:00:29.275585835Z","status":"Running","status_code":103,"resources":{"containers":["/1.0/containers/docker"]},"metadata":{"fds":{"0":"1e7d2045a84d4e19d4e3372f84eeaa58bb8c29d29b40e4ed14ab4043a421db69","control":"09e3d36b9860f9b29e7290c52899f8983eaa30334b55ea5f43f0974d6947dd86"}},"may_cancel":false,"err":""},"operation":"/1.0/operations/357ea5c2-98b6-4ece-9bc8-a82b2fb2c196"}

                                                                                docker: Error response from daemon: linux runtime spec devices: lstat /dev/.lxc/proc/1482/fdinfo/12: no such file or directory.
DBUG[01-27|04:00:30] got message barrier
                                         DBUG[01-27|04:00:30] 1.0/operations/357ea5c2-98b6-4ece-9bc8-a82b2fb2c196/wait
                                                                                DBUG[01-27|04:00:30] Raw response: {"type":"sync","status":"Success","status_code":200,"metadata":{"id":"357ea5c2-98b6-4ece-9bc8-a82b2fb2c196","class":"websocket","created_at":"2017-01-27T04:00:29.275585835Z","updated_at":"2017-01-27T04:00:30.378503307Z","status":"Success","status_code":200,"resources":{"containers":["/1.0/containers/docker"]},"metadata":{"return":127},"may_cancel":false,"err":""}}

Sorry, to clarify: I switched to a container where the docker installation wasn't broken

I'm trying to reproduce it and got similar error

@niko-daniel i'm not too familiar with the code yet unfortunately. Do you think this issue is more appropriate for the https://github.com/lxc/lxc repository?

@CarltonSemple no, this is done using LXD so this repository is the right place for this report.

Sounds like Docker is confused by the mount table in the LXD container. We'll have to take a look and may have to file an issue with Docker upstream for this.

@mwhudson @brauner does that ring a bell with either of you?

No. But it's trivial to reproduce:
$ lxc exec docker -- docker run --privileged hello-world
docker: Error response from daemon: linux runtime spec devices: open /dev/.lxc/proc/145/fdinfo: permission denied.
ERRO[0000] error getting events from daemon: net/http: request canceled

This turns out to be with the docker upstream 1.13.0-rc1 package installed in the container, because that's what I happened to have lying around. Can try with newer or Ubuntu packaging but somehow doubt that's going to make a difference.

On 30 Jan 2017 07:01, "Michael Hudson-Doyle"<EMAIL_ADDRESS>wrote:
Can try with newer or Ubuntu packaging but somehow doubt that's going to
make a difference.
It doesn't.

Just so you know, I had another issue that is probably related. (in LXD container), Docker pull of some images doesn't complete: https://github.com/docker/docker/issues/30569

Hi @stgraber, do you have any temporary workarounds, or a rough estimate of when this might be worked on?

Or do you have any pointers on where to look so I could help?
@stgraber

Hmm, so I'm not sure there's an easy workaround for this other than having Docker fixed to not use paths it can't access...
/dev/.lxc/proc and /dev/.lxc/sys are paths mounted by LXC to allow for container nesting in environments where the kernel would deny mounting of new proc and sys instances due to the overmounting protection kicking in. Unmounting those would prevent spawning nested containers and allowing the read and writes under those paths would be a potential security issue.
The correct fix here is to fix Docker with one of:

Just assume that proc is mounted at /proc and sys at /sys, it's not like they can really be mounted somewhere else and not at those locations without breaking the system.
Keep their current logic but skip /dev/.lxc
Keep their current logic and skip to the next match on permission denied

@mwhudson sounds like we may want an extra test for --privileged in our autopkgtest and a patch to fix this issue ^

@stgraber Agreed. I'm doing some investigating right now, building docker myself. The error comes when the daemon tries to read process information about itself, in /dev/.lxc/proc/<daemon_pid>.

@CarltonSemple what's odd is that it only does so when --privileged is passed? I'd think that docker would access /proc/self at some other point too... Anyway, I still find it weird that they go through all the trouble of scanning /proc/mounts (or mountinfo) for all instances of /proc rather than just pick the standard path for it...

@stgraber Yes. The function where the behavior changes is setDevices() in  /docker/daemon/oci_linux.go. The code that breaks is actually from the OpenContainers repository in /runc/libcontainer/devices/devices_unix.go.
I was able to add ".lxc" to the skipped directories in the getDevices() function, getting past the "no such file or directory" error. Now it runs into another error. This is the docker daemon output, with some extra debug statements :

INFO[0001] Loading containers: done.
INFO[0001] Daemon has completed initialization
INFO[0001] Docker daemon                                 commit=1564f02-unsupported graphdriver=overlay version=1.12.4
INFO[0001] API listen on /var/run/docker.sock
Privileged...
skipping  .lxc
skipping  .lxd-mounts
appending
appending
/dev / hugepages
skipping  lxd
skipping  mqueue
/dev / net
appending
appending
skipping  pts
appending
skipping  shm
appending
appending
appending
ERRO[0010] containerd: start container                   error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:291: setting cgroup config for ready process caused \"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/997a8faece21427b2efe84c901978b178c3cbcb67da6787c2782ba23750c8a21/devices.allow: operation not permitted\""
 id=997a8faece21427b2efe84c901978b178c3cbcb67da6787c2782ba23750c8a21
ERRO[0010] Create container failed with error: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:291: setting cgroup config for ready process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/997a8faece21427b2efe84c901978b178c3cbcb67da6787c2782ba23750c8a21/devices.allow: operation not permitted\\\"\"\n"
ERRO[0010] Handler for POST /v1.24/containers/997a8faece21427b2efe84c901978b178c3cbcb67da6787c2782ba23750c8a21/start returned error: invalid header field value "oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:291: setting cgroup config for ready process caused \\\"failed to write a *:* rwm to devices.allow: write /sys/fs/cgroup/devices/docker/997a8faece21427b2efe84c901978b178c3cbcb67da6787c2782ba23750c8a21/devices.allow: operation not permitted\\\"\"\n"

Ok, that one is expected too, you can't write to devices.allow or devices.deny from inside a user namespace. I'm guessing Docker already has logic to deal with this somewhere else, otherwise normal containers would fail too.

It looks related to this older issue: https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1593301

Though that was only showing up when the LXD container itself was privileged, that shouldn't happen when the LXD container is unprivileged, unless Docker's --privileged bypasses whatever code was avoid that cgroup config.

Gotcha. So I switched back to an unprivileged LXD container, and that error no longer shows up.
Now I'm getting what seems to be an issue with Docker inside the LXD container not being able to view all of a process's information.
root@docker2:~# ./dockerd-1.13.0
INFO[0000] libcontainerd: new containerd process, pid: 475
INFO[0001] [graphdriver] using prior storage driver: overlay
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support swap memory limit.
WARN[0001] Your kernel does not support cgroup rt period
WARN[0001] Your kernel does not support cgroup rt runtime
INFO[0001] Loading containers: start.
WARN[0001] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module bridge not found in directory /lib/modules/4.4.0-59-generic
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/4.4.0-59-generic
, error: exit status 1
WARN[0001] Running modprobe nf_nat failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.0-59-generic`, error: exit status 1
WARN[0001] Running modprobe xt_conntrack failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module xt_conntrack not found in directory /lib/modules/4.4.0-59-generic`, error: exit status 1
INFO[0001] Firewalld running: false
INFO[0001] Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address
INFO[0002] Loading containers: done.
INFO[0002] Daemon has completed initialization
INFO[0002] Docker daemon                                 commit=49bf474-unsupported graphdriver=overlay version=1.13.0
INFO[0002] API listen on /var/run/docker.sock
WARN[0103] Your kernel does not support swap memory limit.
WARN[0103] Your kernel does not support cgroup rt period
WARN[0103] Your kernel does not support cgroup rt runtime
ERRO[0103] containerd: start container                   error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 631 caused \"readlink /proc/631/fd/0: permission denied\""
 id=b638a5975734e527a8c15fc24b5c6222c8ceced1414c7657a6a4362ebb5ed95c
ERRO[0103] Create container failed with error: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 631 caused \"readlink /proc/631/fd/0: permission denied\""

ERRO[0103] containerd: deleting container                error=exit status 1: "container b638a5975734e527a8c15fc24b5c6222c8ceced1414c7657a6a4362ebb5ed95c does not exist\none or more of the container deletions failed\n"
ERRO[0104] Handler for POST /v1.25/containers/b638a5975734e527a8c15fc24b5c6222c8ceced1414c7657a6a4362ebb5ed95c/start returned error: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 631 caused \"readlink /proc/631/fd/0: permission denied\""

What's weird is that the root user inside of the container can go into the /proc//fd directory, but doesn't seem to have access to all of the files.
root@docker2:/proc/469/fd# ls
0  1  10  11  12  13  2  3  4  5  6  7  8  9
root@docker2:/proc/469/fd# cat 2

^C
root@docker2:/proc/469/fd# cat 4
cat: 4: No such device or address
root@docker2:/proc/469/fd# ls
0  1  10  11  12  13  2  3  4  5  6  7  8  9
root@docker2:/proc/469/fd# cat 5
cat: 5: Permission denied
root@docker2:/proc/469/fd#

That's indeed a bit weird. Can you try to ssh into the container or run docker through a "script /dev/null" session perhaps?
I want to check if maybe the problem is to do with the pts device that you get from LXD when doing "lxc exec" (which coming from the host would be owned by the host and so may cause those permission issues).

@mwhudson sounds like we may want an extra test for --privileged in our autopkgtest and a patch to fix this issue ^

The extra test is trivial, as is adding a patch or two, once we have them...

@stgraber you mean like this? I also tried building off of git checkout tags/v1.12.3 to get closer to the default apt install docker.io, but I think I might have the wrong version. Dockerd was complaining about not being able to connect to containerd, so that's why I'm building off of git checkout tags/v1.13.0.
root@docker3:~# script /dev/null
Script started, file is /dev/null
# ./dockerd-1.13.0
INFO[0000] libcontainerd: new containerd process, pid: 2854
INFO[0001] [graphdriver] using prior storage driver: overlay
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support swap memory limit.
WARN[0001] Your kernel does not support cgroup rt period
WARN[0001] Your kernel does not support cgroup rt runtime
INFO[0001] Loading containers: start.
WARN[0001] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module bridge not found in directory /lib/modules/4.4.0-59-generic
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/4.4.0-59-generic
, error: exit status 1
WARN[0001] Running modprobe nf_nat failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.0-59-generic`, error: exit status 1
WARN[0001] Running modprobe xt_conntrack failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-59-generic/modules.dep.bin'
modprobe: WARNING: Module xt_conntrack not found in directory /lib/modules/4.4.0-59-generic`, error: exit status 1
INFO[0001] Firewalld running: false
INFO[0001] Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address
INFO[0001] Loading containers: done.
INFO[0001] Daemon has completed initialization
INFO[0001] Docker daemon                                 commit=49bf474-unsupported graphdriver=overlay version=1.13.0
INFO[0001] API listen on /var/run/docker.sock
WARN[0042] Your kernel does not support swap memory limit.
WARN[0042] Your kernel does not support cgroup rt period
WARN[0042] Your kernel does not support cgroup rt runtime
ERRO[0042] containerd: start container                   error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 3002 caused \"readlink /proc/3002/fd/0: permission denied\""
 id=96eee0b4ec4e56336f030bc34b881121f1c7c109c13bf2265fa166838a89cb06
ERRO[0042] Create container failed with error: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 3002 caused \"readlink /proc/3002/fd/0: permission denied\""

ERRO[0042] containerd: deleting container                error=exit status 1: "container 96eee0b4ec4e56336f030bc34b881121f1c7c109c13bf2265fa166838a89cb06 does not exist\none or more of the container deletions failed\n"
ERRO[0042] Handler for POST /v1.25/containers/96eee0b4ec4e56336f030bc34b881121f1c7c109c13bf2265fa166838a89cb06/start returned error: oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 3002 caused \"readlink /proc/3002/fd/0: permission denied\""

The error looks like it's coming from https://github.com/opencontainers/runc/blob/master/libcontainer/process_linux.go#L247 , and might be related to https://github.com/opencontainers/runc/issues/1130

@stgraber is the code for the LXD version of Docker on Github anywhere? I want to make sure I'm not running into unrelated errors.

@CarltonSemple I don't believe @mwhudson maintains it in git but you can go to https://launchpad.net/ubuntu/+source/docker.io and grab the debian.tar.xz tarball for the latest release there, you'll find a debian/patches directory which includes all the patches that Ubuntu's docker has on top of the clean upstream tarball.

The packaging is maintained at
https://github.com/tianon/debian-docker/tree/ubuntu (sometimes
https://github.com/mwhudson/debian-docker/tree/ubuntu is more up to date
for a day or two) but that's not really the most comprehensible form if
you're not used to debian packaging already.
On 14 February 2017 at 08:31, Stéphane Graber<EMAIL_ADDRESS>wrote:

@CarltonSemple https://github.com/CarltonSemple I don't believe
@mwhudson https://github.com/mwhudson maintains it in git but you can
go to https://launchpad.net/ubuntu/+source/docker.io and grab the
debian.tar.xz tarball for the latest release there, you'll find a
debian/patches directory which includes all the patches that Ubuntu's
docker has on top of the clean upstream tarball.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lxc/lxd/issues/2825#issuecomment-279496704, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApBFr46lyOlZe_yvUxVErmnPN13T__Aks5rcK93gaJpZM4LvD7i
.

@mwhudson yes, looking through that, I'm not sure of how to make a change to your version of docker and then build it.

Can we confirm that https://github.com/opencontainers/runc/pull/1327 solves the original problem? In any of the docker.io packages at https://launchpad.net/ubuntu/+source/docker.io, the runC code of interest is in the vendor directory of docker. @mwhudson @stgraber

@CarltonSemple
With regards to this issue:

What's weird is that the root user inside of the container can go into the /proc/$(pid)/fd directory, but doesn't seem to have access to all of the files.

That's to be expected. Trying to open and then read from a bunch of random file descriptors a process has open is not always going to work -- what would it mean to read from a socket(2) descriptor for example? Same goes for some pty related file descriptors as well.

@cyphar gotcha. I was able to get past that error though, using an update to runc. I now have Docker 1.13 unprivileged containers running.
Have you been able to verify opencontainers/runc#1327 ? I am not familiar with debian packaging, so the best I could do was download the docker.orig.tar.gz and try building that. opencontainers/runc#1327 gets me past the docker: Error response from daemon: linux runtime spec devices: lstat /dev/.lxc/proc/1482/fdinfo/12: no such file or directory. error.
I was hoping we could at least cover that error in this issue and open another issue for privileged Docker containers.

opencontainers/runc#1327 looks like an okay change to me, and I'll LGTM it once I figure out why the CI broke (it's probably not because of your change). It should fix your issue with Docker, but I would recommend verifying that yourself (try replacing docker-runc with a new runC build).

@cyphar CI failure is related to https://github.com/opencontainers/runc/pull/1237

@cyphar yes, I included the opencontainers/runc#1327 change when building dockerd, and that fixed it without needing to replace docker-runc. The error comes from dockerd using runC code.
As for docker-runc's error error=oci runtime error: container_linux.go:247: starting container process caused "process_linux.go:252: getting pipe fds for pid 3002 caused \"readlink /proc/3002/fd/0: permission denied\"", you may find the patch in this ubuntu discussion helpful. The patch there looks related to what @hqhq mentioned

@stgraber does that last error look like more of an LXD issue?

@CarltonSemple not sure, it says it's creating devnice nodes, but the error suggests it attempted to open it for read/write
/dev/tty in LXD containers should work fine so I'm not sure why the one inside the docker container doesn't.
Can you confirm that "cat /dev/tty" in your LXD container works properly (that is, doesn't print an error and hangs there)?

@CarltonSemple I would very strongly recommend using the line you linked -- it makes runC vulnerable to race conditions that allow processes to access the host (see CVE-2016-9962). I'm working on making this code safe and work properly -- please look at opencontainers/runc#774 and see if you can cherry-pick the changes to nsexec.c and process_linux.go (though they are substantial).
As for the creating device nodes issue, I have the feeling that error you're seeing (ENXIO) is something coming from overlayfs. I'm not sure exactly where, you'll have to debug your setup yourself.

@brauner Since you wrote the patch I would recommend reading the above. The stuff I mentioned at FOSDEM about PRCTL_SET_DUMPABLE with rootless containers have been fixed in opencontainers/runc#774. I can help you split out the key  ^^

This issue was noted in a LXC/LXD presentation at 2017 KubeCon Berlin.

opencontainers/runc#774 has been merged and from what I last heard, @brauner is going to backport or otherwise apply opencontainers/runc@6bd4bd9030a979d27323ee4f1c4c5dd44469dc45 (which should fix the unprivileged problems noted above).

Yes, I'm not the one directly responsible for this but @mwhudson does have a plan. :)

Can I get a test case for the unprivileged problems please?
And am I correct to think that there is no pending fix for "lxc exec docker -- docker run --privileged"?
(Sorry, I find this report pretty confusing)

@mwhudson I believe https://github.com/opencontainers/runc/pull/774 is supposed to fix some of the issues, right @cyphar ?
I will check for myself soon

Please can someone help me with the steps to fix the docker: Error response from daemon: linux runtime spec devices: lstat /dev/.lxc/proc/1482/fdinfo/12: no such file or directory. error.
I read that I have to use a specific runC binary

@infinitydon For now, stick to the docker.io package from Ubuntu, not the one you download from Docker upstream otherwise things won't work.

I'm closing this issue because there's nothing for LXD to do here.
That's not to say we won't be doing anything. @mwhudson is still looking at including @cyphar's fixes in the Ubuntu package to improve over our current workaround. @cyphar's work upstream should also ensure less regression of Docker in LXD and hopefully means that the next upstream Docker build will work unmodified in LXD containers.
And we plan on adding daily testing of the dev branch of Docker to detect regression before they hit users.

@infinitydon For now, stick to the docker.io package from Ubuntu, not the one you download from Docker upstream otherwise things won't work.

@stgraber , the same error appears with docker.io=1.12.6-0ubuntu1~16.04.1, runc=1.0.0~rc2-0ubuntu2~16.04.1: while using kubeadm the kube-proxy can't start because of
Error response from daemon: {"message":"linux runtime spec devices: lstat /dev/.lxc/proc/14326/fdinfo/12: no such file or directory"}

Also running into this issue as I'm attempting to run Kubernetes inside of LXD containers.
It appears there may be a fix in 17.06 docker, but that hasn't hit the edge repo yet:
https://github.com/moby/moby/issues/32968

Kubernetes hasn't officially released support for docker > 1.13.x. that I'm aware of. That may change with the 1.7 release, so don't think that upgrading to the CE docker flavors of 17.x will be feasible at this time without introducing other unforeseen problems.

Ahh, one step forward, two steps back.

Hello, I am probably in over my head here.  Trying to run docker via LX[CD].  Previous it was mentioned to use docker.io versus other.  That did not work here.  I am on a Mac running this in a VM (Fusion).  Any tips would be great.  I am trying to run Swarm across several LXD instances in a VM.
user@ubuntu:~$ lxc exec nodef -- sh -c 'apt-get update && apt-get install docker.io && docker version && docker run hello-world'
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
...
Get:42 http://archive.ubuntu.com/ubuntu xenial-backports/universe Translation-en [2872 B]
Fetched 23.9 MB in 53s (442 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  bridge-utils cgroupfs-mount containerd runc ubuntu-fan
Suggested packages:
  mountall aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils
The following NEW packages will be installed:
  bridge-utils cgroupfs-mount containerd docker.io runc ubuntu-fan
0 upgraded, 6 newly installed, 0 to remove and 20 not upgraded.
Need to get 16.4 MB of archives.
After this operation, 83.6 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu xenial/main amd64 bridge-utils amd64 1.5-9ubuntu1 [28.6 kB]
...
Get:6 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 ubuntu-fan all 0.9.2 [30.7 kB]
Fetched 16.4 MB in 25s (644 kB/s)
Selecting previously unselected package bridge-utils.
(Reading database ... 25504 files and directories currently installed.)
Preparing to unpack .../bridge-utils_1.5-9ubuntu1_amd64.deb ...
Unpacking bridge-utils (1.5-9ubuntu1) ...
Selecting previously unselected package cgroupfs-mount.
Preparing to unpack .../cgroupfs-mount_1.2_all.deb ...
Unpacking cgroupfs-mount (1.2) ...
Selecting previously unselected package runc.
Preparing to unpack .../runc_1.0.0~rc2-0ubuntu2~16.04.1_amd64.deb ...
Unpacking runc (1.0.0~rc2-0ubuntu2~16.04.1) ...
Selecting previously unselected package containerd.
Preparing to unpack .../containerd_0.2.5-0ubuntu1~16.04.1_amd64.deb ...
Unpacking containerd (0.2.5-0ubuntu1~16.04.1) ...
Selecting previously unselected package docker.io.
Preparing to unpack .../docker.io_1.12.6-0ubuntu1~16.04.1_amd64.deb ...
Unpacking docker.io (1.12.6-0ubuntu1~16.04.1) ...
Selecting previously unselected package ubuntu-fan.
Preparing to unpack .../ubuntu-fan_0.9.2_all.deb ...
Unpacking ubuntu-fan (0.9.2) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for systemd (229-4ubuntu17) ...
Setting up bridge-utils (1.5-9ubuntu1) ...
Setting up cgroupfs-mount (1.2) ...
Setting up runc (1.0.0~rc2-0ubuntu2~16.04.1) ...
Setting up containerd (0.2.5-0ubuntu1~16.04.1) ...
Setting up docker.io (1.12.6-0ubuntu1~16.04.1) ...
Adding group `docker' (GID 116) ...
Done.
Setting up ubuntu-fan (0.9.2) ...
Processing triggers for systemd (229-4ubuntu17) ...
Processing triggers for ureadahead (0.100.0-19) ...
Client:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   78d1802
 Built:        Tue Jan 31 23:35:14 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.6
 API version:  1.24
 Go version:   go1.6.2
 Git commit:   78d1802
 Built:        Tue Jan 31 23:35:14 2017
 OS/Arch:      linux/amd64
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
78445dd45222: Pull complete
Digest: sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7
Status: Downloaded newer image for hello-world:latest
container_linux.go:247: starting container process caused "process_linux.go:359: container init caused \"rootfs_linux.go:42: preparing rootfs caused \\\"permission denied\\\"\""
docker: Error response from daemon: containerd: container not started.
user@ubuntu:~$ lxc exec nodef -- sh -c 'docker run --privileged hello-world'
docker: Error response from daemon: linux runtime spec devices: open /dev/.lxd-mounts: permission denied.
user@ubuntu:~$

I'm sorry to hijack this issue, but do we have some patch to fix this error?
oci runtime error: container_linux.go:262: starting container process caused "process_linux.go:339: container init caused \"rootfs_linux.go:69: creating device nodes caused \\\"open /var/lib/docker/vfs/dir/6bfe63ddb9d78aa0f53bbf7e3de31271c321a052cb2559fa2275d94c40166997/dev/tty: no such device or address\\\"\""

I got this error, exit code 1.
I setup a lib-virt lxc with Debian sid and installed docker.io.


SDN-3979: E2E extend testing by including disruption test cases

- What this PR does and why is it needed
this is follow up to PR #221

/test unit-test

/retest

/retest

/retest

/retest

/retest


undefined method `timeout=' for Rack::Timeout:Class (NoMethodError) for <IP_ADDRESS>

Updating from

rack-timeout (0.4.2)

rack-timeout (<IP_ADDRESS>)

There is an error at
# config/initializers/timeout.rb
Rack::Timeout.timeout = 30  # seconds

The error is
config/initializers/timeout.rb:2:in `<top (required)>': undefined method `timeout=' for Rack::Timeout:Class (NoMethodError)

Looked at the change log, but this breaking change was not mentioned? Is this expected as a breaking change between these two versions.

Yes, this is one of the legacy class setters that was removed in this release.

What are we suppose to use instead? @wuputah

I suppose this -
Rails.application.config.middleware.insert_before Rack::Runtime, Rack::Timeout, service_timeout: 5

It should have the same effect right?

Yes, that works. Alternatively, you can use the environment variables (from #105).

I suppose this -
Rails.application.config.middleware.insert_before Rack::Runtime, Rack::Timeout, service_timeout: 5

It should have the same effect right?

Where should this command be entered? In Heroku CLI?


Duplicate Dialogue System.

it seems you made a mistake when re structuring your project, because there is a second dialogue system inside the dialogue system that causes namespace and declaration bug.

Thanks for the heads up, I'm gonna look into this!


MixedVariableSampling does not work with NelderMead (max_alpha)

I think I found an issue when using NelderMead with MixedVariableSampling. Let's say we have one integer and one real parameter. We could write something like this:
sampling= MixedVariableSampling(
    ["a", "b"], {"a": get_sampling("int_random"), "b": get_sampling("real_random")}
)

these samples are combined internally in the concatenate_mixed_variables function. This function combines them in a new ndarray with the np.object dtype.
NelderMead calls max_alpha (in vectors.py), there pymoo check np.isnan(val) but val's dtype is np.object and the check fails.

Similar issues would also occur in numeric.py for isfinite checks

Thanks for using pymoo! NelderMead ONLY works for continuous variables (by the design of the algorithm).
Does GA work for you?


Compatability: many style edits based on jnewbery feedback

Based on @jnewbery feedback via email and https://github.com/harding/bitcoinops.github.io/pull/6

Looks much better. Thanks @harding
ACK 3d9406e253bb4c45578c0d26f92ff72a15806a52

tACK, thanks for doing the legwork here @harding !


Crypto - Showing as red (percentage and arrow) when up

Ticker = BITCOIN
Type = Crypto
Version: Latest from github
Bitcoin is up in the last 24hrs, but percentage showing as RED with RED down arrow.
Additional ask, GREEN arrow (in general) should point UP to the RIGHT. Currently pointing UP to the LEFT.

Updated to 1.1.3 and it resolved the issue.


Thanks a lot for logtrail

It's the one of the best apps available for Kibana. Nice work!
The only thing I'm missing from Papertrail is being able to share links on specific timestamps to other team members.

Thanks for your kind words. There are multiple issues related to this #255 and #206.


feat: Assistant stream mode and event callback

This update enhances the existing functionality by adding support for Assistant stream mode and stream event callbacks without introducing any breaking changes or modifications to the existing API endpoints.
The implemented features are in line with the OpenAI Assistant streaming API, as documented here: OpenAI Assistant Streaming API
Stream.On("event1", callback1)
Stream.On("event2", callback2)
Stream.Run()

Thank you for the PR! Sorry for a long review process :D


Add new role: EvmRole-user_global_region

@gtanzillo @Loicavenel Can you please review the list of enabled product features is OK? Thanks.
https://bugzilla.redhat.com/show_bug.cgi?id=1678142

@mzazrivec I don't see any issues with this new role. But I do have a couple of questions -

Did you create the new role by copying and modifying an existing one? This looks similar to EvmRole-user
Are there a set of product features in the new role that distinguish a user as a "global" user or is it just in the name? I don't see anything jumping out at me.

@mzazrivec Looks like you need this change to update the rolde counts and make the specs pass -
diff --git a/spec/models/miq_user_role_spec.rb b/spec/models/miq_user_role_spec.rb
index 4146bb32dd..7c67440036 100644
--- a/spec/models/miq_user_role_spec.rb
+++ b/spec/models/miq_user_role_spec.rb
@@ -1,6 +1,6 @@
 describe MiqUserRole do
   before do
-    @expected_user_role_count = 19
+    @expected_user_role_count = 20
   end

   context ".seed" do

@gtanzillo yes, I created the new role from EvmRole-user and removed some features as instructed in https://github.com/ManageIQ/manageiq-ui-classic/pull/5634#issuecomment-500397124

# select pf.identifier product_feature from miq_user_roles ur, miq_roles_features rf, miq_product_features pf where ur.id = rf.miq_user_role_id and rf.miq_product_feature_id = pf.id and ur.name = 'EvmRole-user' except select pf.identifier product_feature from miq_user_roles ur, miq_roles_features rf, miq_product_features pf where ur.id = rf.miq_user_role_id and rf.miq_product_feature_id = pf.id and ur.name = 'EvmRole-user_global_region';
        product_feature
-------------------------------
 vm_drift
 miq_template_drift
 vm_tag
 miq_template_show
 vm_perf
 vm_check_compliance
 miq_template_compare
 miq_template_check_compliance
 miq_template_tag
 vm_snapshot
 miq_template_perf
 cockpit_console
 chargeback_reports
 vm_webmks_console
 vm_vmrc_console
 miq_template_snapshot
 miq_template_timeline
 all_vm_rules
 vm_show
 vm_console
 vm_timeline
 vm_vnc_console
 vm_show_list
 miq_template_show_list
 vm_chargeback
 vm_compare
(26 rows)

@gtanzillo Updated the spec, thanks!

@gtanzillo : Are you happy with the list of features (can I merge this)?


Checks for WebGL constraints

Big programs with too much workers or too much variables fail silently because of WebGL texture size and texture amount boundaries. It would make sense to check for them and throwing exceptions to make debugging easier.

That wouldn't be hard to implement. On the long term, something even better would be to pack multiple arrays on the same texture. Currently, one must do it himself, i.e., rather than allocating 10 different arrays of size 100, alloc one of size 1000.


Controller failing to associate lattice service with service network in different account.

I have a VPC Lattice Service Network ('my-hotel') defined in AWS Account A. This is RAM-shared to Account B, where I am running my EKS cluster. I can see the Service Network when logged into the console and cli of Account B. I can successfully associate my EKS VPC with the 'my-hotel' service network.
Following the Quick Start guide, I deploy the 'parking' and 'review' kubernetes deployments and services into my cluster, and then the 'rates' HTTPRoute resource using the rate-route-path.yaml in the example folder. This creates the 'rates-default' lattice service and two target groups - 'k8s-default-review-qmsuilsxyq' and 'k8s-default-parking-koyqfohudh'. These are visible in the console of Account B, but not visible from Account A.
The controller fails to associate the 'rates-default' service with the 'my-hotel' Service Network. With debug logging on, the controller logs: -
{"level":"debug","ts":"2023-12-08T11:03:40.393Z","logger":"cloud","caller":"aws/cloud.go:65","msg":"response","serviceName":"VPC Lattice","operation":"ListServiceNetworks","params":"{\n\n}"} {"level":"info","ts":"2023-12-08T11:03:40.394Z","logger":"controller.route","caller":"controllers/route_controller.go:155","msg":"reconcile error","name":"rates","message":"error during service synthesis failed ServiceManager.Upsert rates-default due to Service network my-hotel not found"}
I am able to manually associate the 'rates-default' service with the 'my-hotel' service network via the console, logged in to Account B.
Have I missed something to allow the controller to create lattice services / target groups and associate with a lattice service in a different aws account?

There is confirmed issue on controller/lattice side with RAM. It might be your case, but I recommend double check on your side.
There was recent change on controller side to use tags for ownership for shared resource like SN. When SN is shared with you through RAM there is no option to allow ListTagsForResource.
aws vpc-lattice list-tags-for-resource --resource-arn ${shared_sn_arn}
This call will fail, even if you grant "full" permission in RAM AWSRAMPermissionVpcLatticeServiceNetworkReadWrite
You should see something like this in debug or error logs:
An error occurred (AccessDeniedException) when calling the ListTagsForResource operation: User: arn:aws:sts::Redacted:assumed-role/Redacted is not authorized to perform: vpc-lattice:ListTagsForResource on resource: arn:aws:vpc-lattice:us-west-2:Redacted:servicenetwork/sn-Redacted
We are looking into solutions.

it should be fixed now and will be released in v1.0.2 soon

please try new release, closing ticket for now

Thank you @mikhail-aws . Yes, I can confirm the new release works as desired, and allows the controller to create a service and associate with a service network in a different account.


del mesa-darwin: fossil (2012 version)

Motivation for this change
Main point
(2018-06-29) v18.1.3 - mesa
(2012-10-24) v08.0.3 - mesa-darwin
I do mesa v18 work, and mesa-darwin v8 relies on mesa and creates conflicts during evaluation.
Good point is - from 2012 to this date it was evaluating - that shows the stability achieved by the Nix approach.
###### Serious jokes
I really can not test mesa-darwin v8, both because of architecture, and because it is mesa in 2012 edition style.
I summon the @jtojnar - daedric prince! One of your spheres of influence is guiding forces to cleanse the old, so new comes to the world.
###### Fun time
I, not once, checked everything. Now is the only time I probably ever give myself that fun:
- [x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
- Built on platform(s)
_   - [x] NixOS_
_   - [x] macOS_
_   - [x] other Linux distributions_
- [x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
- [x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
- [x] Tested execution of all binary files (usually in ./result/bin/)
- [x] Determined the impact on package closure size (by running nix path-info -S before and after)
- [x] Fits CONTRIBUTING.md.
Yes! I did it! 8)
Seriously, it conflicts with mesa.

You seem to have just removed the expression without removing the package calls.
$ rg libGLDarwin
pkgs/top-level/all-packages.nix
10888:  libGL = libGLDarwinOr mesa_noglu.stubs;
10891:  libGLU = libGLDarwinOr mesa_glu;
10897:  libGLU_combined = libGLDarwinOr (buildEnv {
10904:  libGL_driver = libGLDarwinOr mesa_drivers;
10908:  libGLDarwin = callPackage ../development/libraries/mesa-darwin {
10914:  libGLDarwinOr = alternative: if stdenv.isDarwin then libGLDarwin else alternative;

Also I would like to hear from @cstrahan or someone if this relict is still needed.

Yes we need to get latest mesa to build on macOS before we can start doing this.

@matthewbauer
Please say more.
I don't understood the message: "we need to get latest mesa build on macOS before we can start" deleting it?

I don't understood the message: "we need to get latest mesa build on macOS before we can start" deleting it?

Yes - the problem right now is that mesa-darwin is still used in a few places (mostly xorg stuff). It definitely should be removed because it is unmaintained & out of date but we don't have a good alternative yet.

Ok. In fact I can avoid this conflict, by not cleaning the config flags.

I would submit pull request mesa that would work mesa-darwin, commenting legacy stuff.

Then would open issue for mesa-darwin, because there is none at the moment.

Closed in favor of both: 1) #43378, 2) seems still used to some degree.


daedalus address to bytes

Hello. Could you point me on how to generate bytes from daedalus address?
DdzFFzCqrht9W56zJGEFvHHywdeXZiGVYGqVhoZj6SRrS9o2HNLmorEzZhKm7khqfBKvCaTKGLtTnQSToxuvdzJTkQqcAf6f2ErxbSKS -> byte[]

Has this been hashed? Can you give some context about the encoding? I can't quite figure out how you ended up with this base64-looking string.

Is this a Byron Address?

Has this been hashed? Can you give some context about the encoding? I can't quite figure out how you ended up with this base64-looking string.
the naive approach would be to simply convert it to bytes:
var bytes = Convert.FromBase64String("Ddz...");
Console.WriteLine(bytes.ToStringHex());

yeah, this worked for me.


[stable/unifi] Fix typos/formatting in README.

What does this PR do?

Fixes typo: Ubiqiti => Ubiquiti
Consistent terminal punctuation.
80 column line breaks.
Converts links to "reference" MD links as generated by Gruber's script.
Cleans up table formatting.

/assign @billimek

/lgtm

Thanks @billimek! 👍


filterPattern to hide

Hi
I am trying to use filterPattern to hide from the list view. Default, it displays only what is filtered but now I would like it to hide the filtered and display the rest. Is there a get around for this?
Thanks!!

Hi,
There's 2 way you could achieve that :

You could use negative look-ahead in your regex (pattern: "(?!yourPattern)")
or you could checkout the feature/filters-and-sorters branch of this repo and use a RegexpFilter with inverted: true :

import SortFilterProxyModel 0.2
//...

    SortFilterProxyModel {
        id: proxyModel
        sourceModel: yourSourceModel
        filters: RegexpFilter {
            roleName: "yourRoleName"
            pattern: "yourPattern"
            inverted: true
        }
    }

Cheers !

Thanks!!
I have used the second way and it works. I have one more issue. I have used a section delegate header component as described here!.  These are grouped based on the role I am using in the filter. When I use the model with the req filter, it removes the heading also which is essential for my work. I am finally trying to achieve some thing like this. Any better implementation/ suggestions are appreciated.

I don't really get what you are trying to do.
Are you trying to hide specific sections with a SortFilterProxyModel ?
The ListView can only show sections it is aware of, so if no element corresponding to a section are present in a model, the ListView won't be able to show the section.
In your screenshot, the datas are quite heterogeneous, I don't think it makes sense to put all them in the same model.
The example code linked makes more sense, but I won't use a proxy model to hide sections here if that's what you want to do. I would just set a flag for each section to hide the elements corresponding to this section but keep the section delegate visible.

Ya I jus wanted to give this a shot if it can be done via SortFilterProxyModel. I will do with the usual flags. Thanks.


Create a Response Model for API Responses

Create a standardized wrapper model for API responses to ensure consistent response structures across all endpoints in the application. This will enhance the readability and usability of API responses for clients consuming the service.
Tasks:

Create a ResponseModel class in the com.mithril.chatapp.common package.
Include fields for:

status (e.g., success, error)
message (string to describe the response)
data (generic type for holding any data)
timestamp (to include the response time, preferably in ISO-8601 format).

Add a utility method for constructing the ResponseModel object easily, e.g., success and error methods.
Integrate the ResponseModel in at least one existing controller to demonstrate its usage.
Update documentation or add JavaDoc comments for better code maintainability.

Example Structure:
public class ResponseModel<T> {
    private String status;
    private String message;
    private T data;
    private String timestamp;

    // Getters and setters

    // Constructors (default and parameterized)

    // Static factory methods for success and error responses
    public static <T> ResponseModel<T> success(T data, String message) {
        // Implementation
    }

    public static <T> ResponseModel<T> error(String message) {
        // Implementation
    }
}

Acceptance Criteria:

The ResponseModel class is implemented with all specified fields and utility methods.
Unit tests are written for the utility methods to verify their correctness.
At least one endpoint returns a response wrapped in the ResponseModel.

Additional Notes:

Follow project conventions for package structure and code style.
Ensure compatibility with JSON serialization by adding necessary annotations (e.g., @JsonInclude, @JsonProperty if using Jackson).

Priority: Medium

Response model is created for userservice


redirect()->withErrors(); not populating $errors {Context: quickstart-basic, Framework Ver. 5.3.0-Dev}

Works with framework ver. 5.2.20.
The validator is failing [good], and the redirect is redirecting to "/" [good], but $errors is not getting the input from withErrors($validator) [bad].
The var_dump($errors); in the view after the validation fail redirect is:
object(Illuminate\Support\ViewErrorBag)[172]
  protected 'bags' =>
    array (size=0)
      empty

Expecting (xdebug set xdebug.max_nesting_level = 512 so I don't know why it's truncated):
object(Illuminate\Support\ViewErrorBag)[135]
  protected 'bags' =>
    array (size=1)
      'default' =>
        object(Illuminate\Support\MessageBag)[136]
          protected 'messages' =>
            array (size=1)
              ...
          protected 'format' => string ':message' (length=8)

Are you sure you upgraded correctly? Our upgrading docs are still only partial at this stage.

The 5.3.0-Dev was a fresh install. The 5.2.20 version I'm contrasting with is a git clone from the quickstart-basic repo. The upgrade guide only mentioned database issues between 5.2 and 5.3.

There is actually an oblique mention of this in the quickstart basic. It says:

Note:
If your copy of Laravel has a RouteServiceProvider that already includes the default routes file within the web middleware group, you do not need to manually add the group to your routes.php file.

It might be helpful to be more specific for people new to the framework. I.e.

Note:
If your copy of Laravel has an app/Providers/RouteServiceProvider.app that already defines the "web" middleware group routes for the application, you do not need to manually add the group to your routes.php file.


 In the past people pipe them into distro and instantly ruin the round.
To fix this we have a two prong approach:

Create a CO2 recycling device to return some of the spent O2 and a magic waste product (or non) at the cost of medium to high power draw.
Add the magnetic gas storage device (SMES for gas) and have this purchased from cargo for a high price or found on salvage wrecks.

A proper atmos setup should have minimal gas loss anyway so using cargo should be a last resort. We can then relegate gas miners to rare use cases and late game tech. In future gas miners will also be very high power draw too so this adds alternatives.

"there is Sabatier reaction which takes in place 200–300 degree celcius, which is;
CO2+2(H2)→ O2+CH4
This reaction is also used in ISS for O2 source and by product of methane."
We'd need methane and hydrogen, or alternatively, plasma, I guess, gas and methane could be used for welding fuel or jetpacks, etc.

we have tritium which is just H3, we could do some magic space jumbo

Yeah we bikeshedded this in discord and came up with just magic the hydrogen away. I think there’s little merit in taking up our 8th gas slot with it unless it’s also used for stuff like fusion and funny bombs but it would need a non sabotage use too.

Sometimes they might put out a lot, and sometimes they may not put out enough.
Combined with powered recyclers I think that would be a good way of balancing them that isn't just a strict power requirement.

2CO₂ + 4T → O₂ + 2C + 2T₂O
This would be a theoretically viable modified Sabatier reaction that combines 2 carbon dioxide molecules with 4 tritium molecules to form oxygen, 2 carbon atoms, and 2 superheavy water molecules (which could be faked with radioactive water).

Realistically I imagine that gas miners would put out randomly fluctuating amounts, especially if there are multiple of them mining in the same spot. Sometimes they might put out a lot, and sometimes they may not put out enough.
Combined with powered recyclers I think that would be a good way of balancing them that isn't just a strict power requirement.

We bikeshedded some more and came up with the following solutions:

make gas miners output far less gas and cap their max pressure they work in to like 3500kpa
Still do the bigger gas storage tanks that can be moved around as a supplement for ships etc.
Possibly have pressure release valves that can be put on the network (constructed) to prevent over pressure.
Gas recycler will work too, only issue with Ikes suggestion is Tritium is pretty rare at present and highly bomb able.

Proposed gas recycler in #9934.

Closing this unless maintainers / PMs decide getting rid of them is still a good idea.
Only thing missing IMO is a way to purchase larger volumes of air (magnetic storage units) or produce it from ice rocks. And or the red reserve gas tanks that can't be built or moved.


Add share button

Add button to the Now Playing screen to allow sharing of the current song & station via UIActivityViewController

Thanks Chloe. Per the contribution guidelines in the Readme for new features:
Please create a separate branch (e.g. features/3dtouch). Please do not commit on master. If you'd like to create a new branch and resubmit a PR, I will be happy to take a look. Thanks!


OTG / USBGadgets support

Implement the Linux gadget modules for g_mass_storage
See: https://github.com/danmons/retronas/discussions/287
Bus 003 Device 020: ID 0525:a4a5 Netchip Technology, Inc. Linux-USB File-backed Storage Gadget

Pi Zero W detected on PC (Linux/Win) but not on Wii nor PS3.
Powered USB hub makes no diference, currently implemented with a 30GB iSCSi LUN (block level) formatted as FAT32

this works on PS3 (formatted MacOS 9.2)(
sdd
└─sdd1      vfat   FAT32 UNTITLED  AD39-0000

seems like there was an issue with the initial format
rebuilt the device on linux, formatted lun as vfat type c, rebuilt on windows as well everything is OK working on PS3 but a powered usb hub is required

device is also recognized on the wii in some homebrew, but not the main use cases
Works:

Homebrew menu
WiiXplorer

Not Working

USBloader
WiiFlow.

Can't cleanrip from Wii to the gadget (iSCSI mounted lun) need to look into that oneday

timeouts are occurring but this is related to iSCSI so needs to be looked at separately from the gadget implementation
[  235.986871] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 4bc35ffa ep0, 0)
[  235.986908] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt: can't clear halt on ep0
[  236.149504] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 76a8ca4e ep1in, 0)
[  236.159788] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 12e95dd9 ep1out, 0)
[  249.303098] dwc2 20980000.usb: new device is high-speed
[  249.503373] dwc2 20980000.usb: new address 1
[  249.563618] dwc2 20980000.usb: new device is high-speed
[  249.763987] dwc2 20980000.usb: new address 2
[  249.764174] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 4bc35ffa ep0, 0)
[  249.764207] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt: can't clear halt on ep0
[  250.123031] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 76a8ca4e ep1in, 0)
[  250.133273] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 12e95dd9 ep1out, 0)
[  254.318595] dwc2 20980000.usb: new device is high-speed
[  254.518920] dwc2 20980000.usb: new address 1
[  254.579184] dwc2 20980000.usb: new device is high-speed
[  254.779513] dwc2 20980000.usb: new address 2
[  254.779710] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt(ep 4bc35ffa ep0, 0)
[  254.779744] dwc2 20980000.usb: dwc2_hsotg_ep_sethalt: can't clear halt on ep0
[ 2110.107919]  connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 179999, last ping 180504, now 181008
[ 2110.108074]  connection1:0: detected conn error (1022)

Seems fairly unstable on the rpi zero w, couldn't say if this would improve on another platform or not

grabbed a USB meter to try and track power draw causing issues on the wii, seems not to be the case just seems writing to the pizero in certain load scenarios is junk

basic OTG support was added in de36c764bca29415f2bd9991f4c1ab6046678a37
user can configure the retronas device to operate in OTG mode by loading the g_mass_storage module with a disk or image (from retronas_path/images) attached. This a bash script executed through the menu to configure the required systemd units
we do (will?) not handle the config.txt modifications required on the rpi to enable the dwc2 module

block device, partitions and cd/dvd/bdrom devices are permitted to be selected.
No idea if a user would want to present a cdrom as a mass_storage device but might be interesting i suppose. could you ODE something with a basic OTG board? ... tis a question for the ages!
intent was to write this so it could be expanded to other gadgets, we may even default to g_multi later on if it behaves (have read it does not with g_serial needing a tty even when it is not used)
Documentation is here: https://github.com/danmons/retronas/wiki/Gadgets


WIP: Type-safe EntityManager wrapping, closes #1036

[x] Have you followed the guidelines in our Contributing document?
[ ] [v2] Have you created a JIRA and used it in the commit message?
[x] [v4] Have you created a GitHub Issue and used it in the commit message?
[x] Have you checked to ensure there aren't other open Pull Requests for the same issue?
[x] Have you built the project locally prior to submission with mvn clean install?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Will go ahead with the approval and the merge later today if no further comments are provided

Adding a hold label since it is WIP

@kg6zvp is this really WIP or can it be merged?

@kenfinnigan It's missing a test and I was going to be out of town so I didn't want to merge to prod without it.

no problem, thanks for the update


Wsl2 support

Hey!
Really like the project and think it's such a cool concept to learn about and build on. Awesome job.
Found this works smoothly on Windows itself, but I had some issues when trying to run omd in a WSL2 environment which this pull request aims to address.
The first issue had to do with opening the browser, where I would get the following error:
Failed to open browser: Os { code: 2, kind: NotFound, message: "No such file or directory" }

The fix for me here was to check if the linux environment we are in is WSL2 (this could be more robust, but this initial check worked for me) and if so, use powershell to start up the browser (cmd was showcasing weird behavior). Otherwise, use xdg-open.
The second had to do with watching the markdown file. At least for WSL2, I noticed that there was an issue with reading the file after making one change where I would then get an Error reading file: No such file or directory (os error 2) error, and any further changes to the markdown file wouldn't seen or updated in the browser. To be honest I see the same thing happen when I run it on my fully linux laptop as well, but for this pull request I just wanted to focus on the WSL2 stuff.
From what I could tell it seemed similar to this issue: https://github.com/notify-rs/notify/issues/254, where they suggested using PollWatcher based on some duration. I found that ended up working for me, and so now if the check for WSL2 is true, omd will use PollWatcher, and RecommendedWatcher otherwise.
Let me know what you think! Happy to change code around to suit what you want. As I mentioned earlier I did notice the watching issue happening on my Linux laptop as well (switching to a PollWatcher helped in that case too), but for this PR I just wanted to address the WSL2 stuff so as not to mess with too many things at once.

Merged.
Thank you @dylanfair for your contribution!


The specified conversion is not valid.

Hi. Install version 19.1.502. I have always used the Xpand.ExpressApp.ImportWizard.
 If you use nuget add our LAB NugetServer as a nuget package source in VS.
Thanks a lot for your contribution.

Closing issue for age. Feel free to reopen it at any time.
.Thank you for your contribution.


[ESLint Plugin] Fix failing tests

PRs for packages that import the plugin will fail because of failures in the plugin test suite after recent upgrades and this PR fixes these issues to some extent.
Two of the failed unit tests were fixed by their respective rule fixer and it is now required that we check the output of the fixer. The other failure is due to an obscure change in behavior after upgrading TS to v4.1.2. I opened https://github.com/Azure/azure-sdk-for-js/issues/13186 to investigate this.

failures in the plugin test suite after recent upgrades

Which recent upgrades?

failures in the plugin test suite after recent upgrades

Which recent upgrades?

@ramya-rao-a I was referring to the version upgrades work done here and here.

@ramya-rao-a I was referring to the version upgrades work done here and here.


Update README to mention Angular SDK

Add comment like what Angular SDK has about AngularJS SDK
Fixes #210

@mraible Good point, thanks! Updated.

LGTM 👍


[Screen Reader-Help-Projects]: Narrator focus not in proper order inside Projects screen in Scan mode.

User Experience:
Users who depends on Screen reader will get confuse if Narrator focus not in proper order inside Projects screen in Scan mode.
Test Environment:
OS: RS2 Version 1703(OS Build 15063.483)
Platform: Edge.
Pre-Requisite: Turn on Narrator
Scan mode : (Caps + Space to set Narrator to scan mode . Use Up/Down arrow keys to navigate )
Repro Steps:-
1- Navigate to https://makecode.microbit.org/acc
2- Navigate to the Microbit section element and select Code control given on it.
3- Navigate to the Help control lying in the header section on the page and select it.
4- Navigate to 'Projects' control and select it.
5- Navigate to various controls lying on the pane opened.
6- Verify that Screen Reader behavior.
Actual Result:-
In Scan mode(CAPS + Space), Narrator focus not in proper order inside Projects screen as it moves like 'Open a documentation in New tab' link -->Projects header
Expected Result:-
In Scan mode(CAPS + Space), Narrator focus should be in  proper order inside Projects Buttons screen as it should move  like Projects header--> 'Open a documentation in New tab' link
Note:
This screen is not supported by NVDA properly
.
Same issue exists in Narrator items mode(CAPS +Arrow keys) as well.
MAS Reference:
https://microsoft.sharepoint.com/teams/msenable/Documents/Forms/AllItems.aspx?id=%2Fteams%2Fmsenable%2FDocuments%2FLCA%2FMAS - July 1%2C 2017 Release%2FMAS 4.2.10 – Tracking of Focus and Selection Attributes.docx&parent=%2Fteams%2Fmsenable%2FDocuments%2FLCA%2FMAS - July 1%2C 2017 Release
Please refer the attachment
MAS4.2.10_Projects_Narrator focus not in order.zip

Verified on URL:  https://makecode.microbit.org/acc and issue is fixed. Hence closing the bug.


[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling.

FIX #9834 (link existing issues this PR will resolve)
Problem
The current BatchExpansionTop1Scorer implements a speculative scoring mechanism that uses batch expansion to estimate the probabilities of speculative tokens based on the scoring model. However, in the existing setup, SequenceGroupMetadata applies default sampling parameters (top_p=1.0, temperature=1.0, repetition_penalty=1.0) when generating target probabilities. According to comments in the code, this choice seems to be made since the sampled tokens are not used directly.
Modification
Although we do not directly sample tokens from the target model while scoring, I believe applying consistent sampling parameters to both draft and target probabilities is essential for accurate rejection sampling. The current implementation uses draft probabilities influenced by sampling (filtered by top_p), while target probabilities are not, leading to a mismatch that could affect scoring accuracy. Because the unsampled target probabilities don’t represent actual usage probabilities, I modified the code to apply the same sampling parameters to both draft and target probabilities for consistency in rejection sampling.
In my experiment, this change resulted in a significant difference in the acceptance rate, as shown in the figures below.
Experiment
Setting

Target Model / Draft Model: llama3-70B / llama3-8B
TP: 4
Devices: A100 * 4
Total number of requests: 500
input length / output length: 1024 / 128
sampling parameter: repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=-1
dataset: c4
batch size: 1
K: # of speculative tokens)

As-Is

K
acceptance rate
system efficiency

1
65.1
82.5

2
63.3
68.3

3
62.4
57.4

To-be (applied in this PR)

K
acceptance rate
system efficiency

1
81.9
91.0

2
80.5
82.3

3
80.2
75.4

@sroy745 @LiuXiaoxuanPKU @njhill
Would you please check this PR related to the sampling process?

We may still want to make and use a (shallow) copy of the sampling parameters with the seed removed in the case a seed is set, to avoid doing seeded sampling for the non-final tokens.

@njhill, I'm curious about the reason why the seed should be removed, especially if it is used for the target model sampling and affects the output token selection when proposals are rejected.

We may still want to make and use a (shallow) copy of the sampling parameters with the seed removed in the case a seed is set, to avoid doing seeded sampling for the non-final tokens.

@njhill, I'm curious about the reason why the seed should be removed, especially if it is used for the target model sampling and affects the output token selection when proposals are rejected.

@joennlae ah sorry, perhaps I misremembered the logic, I didn't think those sampled tokens could end up getting used. I'll check it again but if you're right then makes sense to ignore that seed optimization.

Adding /ready to kick off the tests and verify nothing else fails from this

We may still want to make and use a (shallow) copy of the sampling parameters with the seed removed in the case a seed is set, to avoid doing seeded sampling for the non-final tokens.

@njhill, I'm curious about the reason why the seed should be removed, especially if it is used for the target model sampling and affects the output token selection when proposals are rejected.
@joennlae ah sorry, perhaps I misremembered the logic, I didn't think those sampled tokens could end up getting used. I'll check it again but if you're right then makes sense to ignore that seed optimization.

@njhill
Yeah, I also needed to double-check. I think in this part, we might need to use seeds, but I haven't examined seeded_seqs in detail yet.
https://github.com/vllm-project/vllm/blob/3a763ba0c3a92fdde78e855ded94f9ff29e02088/vllm/model_executor/layers/rejection_sampler.py#L183-L189

We may still want to make and use a (shallow) copy of the sampling parameters with the seed removed in the case a seed is set, to avoid doing seeded sampling for the non-final tokens.

@njhill, I'm curious about the reason why the seed should be removed, especially if it is used for the target model sampling and affects the output token selection when proposals are rejected.
@joennlae ah sorry, perhaps I misremembered the logic, I didn't think those sampled tokens could end up getting used. I'll check it again but if you're right then makes sense to ignore that seed optimization.

@njhill Yeah, I also needed to double-check. I think in this part, we might need to use seeds, but I haven't examined seeded_seqs in detail yet.
https://github.com/vllm-project/vllm/blob/3a763ba0c3a92fdde78e855ded94f9ff29e02088/vllm/model_executor/layers/rejection_sampler.py#L183-L189

Hi,
I think this change should not impact the per request seed handling logic in the RejectionSampler. The per request seeds are set here  which remains unchanged, hence I am wondering if this should be fine.
cc: @tdoublep who made the change for respecting per request seed in spec-decode worker. @tdoublep can you PTAL and see if this change impacts the per request seeding logic or not.
@jeongin601  there is one test failure in the spec_decoding tests (test_many_k[1-32-2-test_llm_kwargs3-baseline_llm_kwargs0-per_test_common_llm_kwargs0-common_llm_kwargs0]). I ran the test locally and it passes. Also from the failure logs it seems transient. Can you please trigger the tests once to see if it passes or not?

Thank you @sroy745, I was able to check correctly after your comments.
I had initially misidentified the part affected by this PR. The 'target_seq_group_metadata_list'(which this PR aims to change) is used here. This section samples 'non_spec_token_ids' (also referred to as retrieved tokens or non-speculative sampled tokens from the target model).
The effect of this PR is to correct the sampling parameter used to generate output tokens from the target model. (These tokens are used when those from the draft model are rejected.) As a result, this PR not only fixes the incorrect values of ‘top_p’ and ‘top_k,’ but also addresses the ‘seeds.'
I also confirmed that this section remains unchanged by this PR and is already using the correct sampling parameters.

test_many_k passed, but test_mlp_e2e_seeded_correctness failed(it didn't raise assertion). I think there shouldn't be any issue with the seed, but we need to check. @jeongin601  will rerun the test first.

@sroy745 I retriggered the tests, but they still fail on a seeded speculative decoding test. It seems this happens because my code reproduces the same results when the sampling seed is set to 'None.' But if the seed value is set to 'None,' sampling should use the default seed value. Then why should it produce different results?

This test was added in this PR.
@njhill Would you please help us to understand the test results? We are confused due to the test failure in 'test_mlp_e2e_seeded_correctness'.

@sroy745 I retriggered the tests, but they still fail on a seeded speculative decoding test. It seems this happens because my code reproduces the same results when the sampling seed is set to 'None.' But if the seed value is set to 'None,' sampling should use the default seed value. Then why should it produce different results?

This test was added in this PR. @njhill Would you please help us to understand the test results? We are confused due to the test failure in 'test_mlp_e2e_seeded_correctness'.

Hi,
I added some more logs to debug the test failure. The test that fails is for temperature 0.1. My understanding is the following.
The output is determined by the probability distribution of the target model. Prior to this change the temperature of the target model would be set to 1 (https://sourcegraph.com/github.com/vllm-project/vllm/-/blob/vllm/spec_decode/batch_expansion.py?L317) and hence the probability distribution would be uniform as shown in the logs below.  This means that the probability distribution of the sampled tokens is no longer uniform (highly skewed). When that happens I think it does not matter if there is a seed or not. We will always sample the token with prob 1.0
=====================================
sorted_target_with_bonus_probs tensor([[[1.0000e+00, 3.2860e-13, 9.0046e-16,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 5.1224e-14, 1.1282e-16,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 5.2276e-07, 8.0635e-08,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 1.0203e-08, 6.8024e-10,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]]], device='cuda:0')
sorted_indices_target_with_bonus_probs tensor([[[29889, 29892, 29991,  ..., 31997, 31998, 31999],
         [  306,    13,     0,  ..., 31997, 31998, 31999],
         [29902,  1576,  4013,  ..., 31997, 31998, 31999],
         [  626,   505,  5360,  ..., 31997, 31998, 31999]]], device='cuda:0')
=====================================
==================
accepted tensor([[ True, False,  True]], device='cuda:0')
substitute_token_ids tensor([[29892,   306, 29902]], device='cuda:0')
==================
=====================================
sorted_target_with_bonus_probs tensor([[[1.0000e+00, 1.4068e-11, 9.6391e-14,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 1.6998e-08, 1.4874e-14,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 1.7947e-14, 4.7633e-18,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [9.9995e-01, 3.7569e-05, 9.5598e-06,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]]], device='cuda:0')
sorted_indices_target_with_bonus_probs tensor([[[  626,   338,  5360,  ..., 31997, 31998, 31999],
         [  263, 29889, 29892,  ..., 31997, 31998, 31999],
         [  304, 29889, 29892,  ..., 31997, 31998, 31999],
         [  367,   304,   748,  ..., 31997, 31998, 31999]]], device='cuda:0')
=====================================
==================
accepted tensor([[ True, False,  True]], device='cuda:0')
substitute_token_ids tensor([[  626,   263, 29889]], device='cuda:0')
==================
=====================================
sorted_target_with_bonus_probs tensor([[[1.0000e+00, 1.1414e-14, 1.0582e-18,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [9.9957e-01, 4.3066e-04, 5.2830e-12,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 9.7951e-11, 4.8004e-12,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00],
         [1.0000e+00, 8.7923e-14, 1.4424e-14,  ..., 0.0000e+00,
          0.0000e+00, 0.0000e+00]]], device='cuda:0')
sorted_indices_target_with_bonus_probs tensor([[[ 7826,  8023, 29889,  ..., 31997, 31998, 31999],
         [  865,   262,  1253,  ..., 31997, 31998, 31999],
         [29889, 12456, 29892,  ..., 31997, 31998, 31999],
         [  306, 22063,  1334,  ..., 31997, 31998, 31999]]], device='cuda:0')
=====================================
==================
accepted tensor([[False, False,  True]], device='cuda:0')
substitute_token_ids tensor([[ 7826,   865, 12456]], device='cuda:0')

I tried running the test with temperature 0.6 instead of 0.1 and it passes.
Not sure if this is correct. @tdoublep / @njhill ptal.

I was always wondering why we use different sampling parameters for the speculative model vs. main model. I reviewed the paper referenced in the rejection sampling code and it doesn't explicitly say either way. I guess one has the freedom to sample the speculative model however one likes, but the results in this PR certainly suggest it makes sense to use the same sampling params as the main model. Really cool! @jeongin601

temperature of the target model would be set to 1 and hence the probability distribution would be uniform

I don't think that is correct: temperature=1.0 doesn't imply a uniform distribution, the distribution is determined by the logits and the temperature is just a scaling factor in the softmax that transforms the logits to probabilities.

However after your change, for the failure case you are now setting the sampling temperature for the target model to 0.1. This means that the probability distribution of the sampled tokens is no longer uniform (highly skewed). When that happens I think it does not matter if there is a seed or not. We will always sample the token with prob 1.0

Yes, setting the temperature to 0.1 will make the test more deterministic than it would be with temperature 1.0. However, it definitely doesn't mean we will always sample the same token with probability exactly 1.0, but rather some high probability.
@sroy745 I think your explanation makes sense: the failing test checks that we get different output each time when not applying the seed. This is not guaranteed to happen when using a low temperature (e.g., approaching the greedy limit), so the test is not really well-defined. The changes in this PR will make everything more deterministic at low temperature, which explains why it was passing before (e.g., because we were using temperature=1.0 to sample the speculative model). I would recommend we modify that test to run only when the temperature is >=1.0.

Thank you for all the reviews and analysis. I understood why my code kept failing the seeded correctness test. To address this, I modified the test to run at temperatures of 0.6, 1.0, and 1.2.

Hi @simon-mo,
The PR has been LG'ed but there is one test failing (distributed-tests-a100). The failure doesn't seem related to this pr. Also confirmed in the #sig-ci channel that this failure is happening in head(https://vllm-dev.slack.com/archives/C07R5PAL2L9/p1732571600143719). If so wondering if we can get this merged?
cc: @LiuXiaoxuanPKU

Thanks @jeongin601 @llsj14 @sroy745 @tdoublep for all of the analysis here!
@jeongin601 could you merge in the latest main which should hopefully address the test failure?


Parse and broadcast incoming trade data from Binance

Allows subscription to optional trades channel on binance and receive public trades feed data
Depends on https://github.com/fremantle-capital/tai/pull/68

@yurikoval can you rebase this one?


MAINT: update version to 0.1.dev0

Update version to 0.1.dev0 to start a "scratch project".

As discussed let's match the version number so that clr version checks will work

Closing this after private discussion with @koubaa. Let us stick with the version of pythonnet.


How to zoom image to crop, not autozoom?

I want zoom image to crop but not autozoom, how to do it? thank for best crop library :D

At the moment only auto-zoom is supported, do you have a scenario where it is not appropriate? what method would you like to use?

why must be autozoom? it really convinient? why not zooming is control by user?

I think it's the preferred UX, that why I implemented it.
I'm not planning to add another zoom mechanism, if you are willing to write the code I will appreciate it.

So Ok, thank u


New endpoints since last update

It appears MailChimp has added some new endpoints since this library's last update. Iw will look into adding support for those shortly.

http://developer.mailchimp.com/documentation/mailchimp/guides/changelog/

Addressed in 3.0.0


Added OSX support to FastImageCache and a demo app

Damn, open since Apr 17? Is this project deprecated?

Nope, not deprecated.

My issue with this PR is that FIC solves an iOS-device-specific problem in a very specific way that is suited to the non-paging-enabled environment of an iOS device. I'm not sure the performance issues that exist on device also exist on the Mac, and I'm even less sure about whether the technique that FIC uses even remotely improves performance on a Mac. If you want to run some tests and show that FIC would be effective on a Mac, I'd be open to it.