Status.UNAUTHENTICATED when trying to set the power limit

Version of the custom_component
Miner
V1.1.0-beta2
HA
Core 2024.1.1
Supervisor 2023.12.0
Operating System 11.3
Frontend 20240104.0
Describe the bug
When trying to set the power limit from within Home Assistant it gets the following error.

log

Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:238
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 16:26:37 (3 occurrences)
Last logged: 16:27:45

[547794924736] (<Status.UNAUTHENTICATED: 16>, 'Missing or invalid authentication token', None)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 238, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2149, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2186, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 272, in handle_service
    return await service.entity_service_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 882, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 952, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/number/__init__.py", line 111, in async_set_value
    await entity.async_set_native_value(native_value)
  File "/config/custom_components/miner/number.py", line 130, in async_set_native_value
    result = await miner.web.grpc.decrement_power_target(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 448, in decrement_power_target
    return await self.send_command(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 326, in send_command
    return (await endpoint(message, metadata=metadata)).to_pydict()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/proto/braiins/bos/v1/__init__.py", line 1837, in decrement_power_target
    return await self._unary_unary(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/betterproto/grpc/grpclib_client.py", line 85, in _unary_unary
    response = await stream.recv_message()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 425, in recv_message
    await self.recv_initial_metadata()
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 393, in recv_initial_metadata
    self._raise_for_grpc_status(status, message, details)
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 345, in _raise_for_grpc_status
    raise GRPCError(status, message, details)
grpclib.exceptions.GRPCError: (<Status.UNAUTHENTICATED: 16>, 'Missing or invalid authentication token', None)

pyasic is trying to use gRPC on the backend to handle setting power limit (as opposed to editing the config, which tends to be really buggy on new models), and for some reason is having issues authenticating with it.  Have you changed the password of that miner at all?
I'll do some testing (either tonight or tomorrow) to see if I have the same issue, but as far as I remember, the implementation was working last time I tried.

I haven't changed the password for this unit, however I can set one and then see if that changes anything.

My testing locally is working good, but that being said, I only have a BCB100, no miner to pair it with, so I'm a f ew version behind, and only able to test with get_ commands, I'm not able to do the increment_power_target command that you're having the issue with.  I would assume the authentication should work the same for all versions and all commands, but I may be wrong.
This may end up being a very complex issue if I'm unable to test on a real miner, but I have asked some friends if they can get me access to some, I'll see if I can get this figured out for you.

So I have set a password on the S19 and still no go. Same error as above.
When i try on an S9 i get the following error
Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:238
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 16:57:43 (1 occurrences)
Last logged: 16:57:43

[547794924736] [Errno 111] Connect call failed ('<IP_ADDRESS>', 50051)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 238, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2149, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2186, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 272, in handle_service
    return await service.entity_service_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 882, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 952, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/number/__init__.py", line 111, in async_set_value
    await entity.async_set_native_value(native_value)
  File "/config/custom_components/miner/number.py", line 134, in async_set_native_value
    result = await miner.web.grpc.increment_power_target(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 435, in increment_power_target
    return await self.send_command(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 319, in send_command
    metadata.append(("authorization", await self.auth()))
                                      ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 333, in auth
    await self._get_auth()
  File "/usr/local/lib/python3.11/site-packages/pyasic/web/bosminer/__init__.py", line 345, in _get_auth
    await stream.send_message(req, end=True)
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 241, in send_message
    await self.send_request()
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 173, in send_request
    protocol = await self._channel.__connect__()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 748, in __connect__
    self._protocol = await self._create_connection()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/grpclib/client.py", line 725, in _create_connection
    _, protocol = await self._loop.create_connection(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1085, in create_connection
    raise exceptions[0]
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1069, in create_connection
    sock = await self._connect_sock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 973, in _connect_sock
    await self.sock_connect(sock, address)
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 628, in sock_connect
    return await fut
           ^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 668, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('<IP_ADDRESS>', 50051)

Ok, that's definitely not supposed to happen.  S9s don't even support gRPC, there's no reason that service should even be getting called in the first place.  I'll take a look through the code tomorrow and see if I can figure it out, but likely need to add a try/except for that auth error.

Ok, that's definitely not supposed to happen. S9s don't even support gRPC, there's no reason that service should even be getting called in the first place. I'll take a look through the code tomorrow and see if I can figure it out, but likely need to add a try/except for that connection error.

@Schnitzel this should be an easy fix, just add ConnectionError to this except line -
https://github.com/Schnitzel/hass-miner/blob/3e8fe0c3891c142b2fb8dab7074b49cbebc0b242/custom_components/miner/number.py#L141
            except (ConnectionError, pyasic.APIError):```

@UpstreamData & @Schnitzel  I can confirm the above code change does work.

Pretty sure I fixed this in v1.1.0, it was more of a pyasic issue, but now that ive seperated S19 and S9s in the backend I think it should work much better.

I don't beleve the component has been actually updated yet, I just manually made the changes to the component on my environment to confirm it worked.

I don't beleve the component has been actually updated yet, I just manually made the changes to the component on my environment to confirm it worked.

FYI, I actually removed all the code that was causing that issue from hass-miner.  It is all handled the way it should have been, with set_power_limit via pyasic, and pyasic seperates between gRPC enabled miners and non gRPC enabled miners.


This attribute was causing a JSON deserialize error for every TPV message,

I removed deny_unknown_fields from the TPVResponse enum. I think this is because of the tag field in the TPV JSON. I noticed that the SkyResponse doesn't deny unknown fields.
Here is an error from before the fix:
raw GPSD data: {"class":"TPV","tag":"RMC","device":"/dev/serial0","mode":3,"time":"2017-09-01T17:48:31.000Z","ept":0.005,"lat":42.250946667,"lon":-83.762138333,"alt":286.300,"epx":25.980,"epy":47.778,"epv":22.770,"track":18.2400,"speed":8.241,"climb":-0.400,"eps":95.56,"epc":45.54}

serde output: Err(ErrorImpl { code: Message("data did not match any variant of untagged enum TpvResponse"), line: 0, column: 0 })
deserializing response failed: ErrorImpl { code: Message("data did not match any variant of untagged enum TpvResponse"), line: 0, column: 0 }
Error(DeserFailed("{\"class\":\"TPV\",\"tag\":\"RMC\",\"device\":\"/dev/serial0\",\"mode\":3,\"time\":\"2017-09-01T17:48:31.000Z\",\"ept\":0.005,\"lat\":42.250946667,\"lon\":-83.762138333,\"alt\":286.300,\"epx\":25.980,\"epy\":47.778,\"epv\":22.770,\"track\":18.2400,\"speed\":8.241,\"climb\":-0.400,\"eps\":95.56,\"epc\":45.54}\r\n", ErrorImpl { code: Message("data did not match any variant of untagged enum TpvResponse"), line: 0, column: 0 }), State { next_error: None, backtrace: None })
raw GPSD data: {"class":"SKY","tag":"GSV","device":"/dev/serial0","xdop":1.73,"ydop":3.19,"vdop":0.99,"tdop":3.67,"hdop":2.44,"gdop":7.11,"pdop":2.64,"satellites":[{"PRN":2,"el":65,"az":283,"ss":16,"used":true},{"PRN":6,"el":60,"az":45,"ss":15,"used":true},{"PRN":12,"el":55,"az":277,"ss":29,"used":true},{"PRN":19,"el":45,"az":104,"ss":19,"used":true},{"PRN":25,"el":27,"az":313,"ss":0,"used":false},{"PRN":17,"el":25,"az":112,"ss":17,"used":true},{"PRN":9,"el":16,"az":82,"ss":0,"used":false},{"PRN":23,"el":11,"az":55,"ss":0,"used":false}]}

Thanks for the PR! Yeah, I'm not entirely sure why deny_unknown_fields was there in the first place - it shouldn't really make a difference in any case. Merging!

Published on crates.io as version 0.4.2.


Explore the use of clone to reduce the number of forks youki does in creating containers

With the release of clone as a new systemcall in kernel version 5.7^, we can now use it to hopefully reduce the number of forks youki does when creating a container (See here).
cc @utam0k @Furisto @flxo

5.7 is indeed too recent, so I think that when we support clone3, we should also support it to work with 5.4 at the same time.

What I got from @flxo's comment on #10, clone2 should give use benefits over the current fork strategy. I say we explore that first.

I have implemented it with the following considerations in mind.
https://github.com/opencontainers/runc/blob/93a01cd4d0b7a0f08abc36c2ebc85a0543b25647/libcontainer/nsenter/nsexec.c#L680-L725
But now youki may get the advantage by using clone(2). I think this may be a challenge. This area has been well commented on by @YJDoc2 . If anyone wants to try it, I will support it.

Just a heads up, I am currently also making changes in this area, so please coordinate with me to avoid stepping on each others toes.

Can I help to take look at this issue? My understanding is if we can enter user namespace and pid namespace together through a single clone(2), the resulting process can then become the init process (YP -> YI, instead of creating YC). We currently have to fork twice because we need to fork once to enter into user namespace, and fork again to enter into the correct pid namespace. Is this understanding correct?

Can I help to take look at this issue? My understanding is if we can enter user namespace and pid namespace together through a single clone(2), the resulting process can then become the init process (YP -> YI, instead of creating YC). We currently have to fork twice because we need to fork once to enter into user namespace, and fork again to enter into the correct pid namespace. Is this understanding correct?

@yihuaf Thanks for your interest. Your understanding is correct. Would you like to take the challenge? However, I think we need to carefully explore whether the double fork had any security implications, and I'd like you to refer to the runc code and check that as well.
https://github.com/opencontainers/runc/blob/93a01cd4d0b7a0f08abc36c2ebc85a0543b25647/libcontainer/nsenter/nsexec.c#L680-L725

Just a heads up, I am currently also making changes in this area, so please coordinate with me to avoid stepping on each others toes.

@Furisto  I'm assuming this is finished, is that correct?

You can assign this to me then. I will keep the referenced runc code in mind.

@yihuaf I'll assign you. I'm looking forward to your PR :)


Strange R^2 value

Hi,
May I ask why the R^2(r-squared) value so small, even negative values existed? This is absolutely strange.

Hi @Luosanj,
please be aware that we only show the training here for 5 epochs. So this model has not converged yet. We usually would train for 1000 epochs and assess convergence during training, so the small $R^2$. value is reasonable.
Please be also aware that the coefficient of determination $R^2$ can in fact be negative as
$$R^2 = 1 - \frac{\sum_i e_i^2}{\sum_i (y_i-\hat{y})^2}$$ for residuals $e_i$ and $\hat{y}$ mean of $y_i$.
For an converged model, the values will increase, but this is not expected after only 5 epochs. Hope this answers your question.

@AnnaChristina  Thanks a lot for your answer ! I am clear now.

Hi @AnnaChristina,
If I want to see the final convergence r-square, how to operate it ?
I have tried several times to set 'epochs = 1000' and run. Yet, the output just shows results of top 30-50  epochs randomly. How to deal with this issue?
Thanks !

This part of ncem is not designed to be run in a jupyter notebook, we just placed it here in order to show an example how you would pass custom deconvoluted data. If you are interested in running more extensive ncem experiments, you may want to check out how to submit train scripts on your compute cluster. Examples for train scripts are provided here: https://github.com/theislab/ncem_benchmarks/tree/main/scripts
Those scripts will save pickle files with training histories and evaluations on train, test and evaluation set within a specified folder, which can be loaded after training.


BrowserStack "localIdentifier" value is not unique enough by default

BrowserStack localIdentifier value is taken from here:
https://github.com/kategengler/ember-cli-browserstack/blob/master/lib/utils/build-name-from-env.js
Same value is used for build property. Problem is that if you for example run a matrix build on Github Actions or any other CI. You would have N number of parallel runs all having the same localIdentifier.
This means that from BrowserStack cloud perspective it has N number of clients all identified as one. Therefore BrowserStack cloud does not know where to route traffic and things just fail.
We can add process.env.BROWSERTACK_LOCAL_IDENTIFIER capability to customise this value OR we can just default to using v4 uuid by default. In general localIdentifier is technical value overall and really not visible anywhere in BrowserStack UI etc.

The localIdentifier is currently passed to both the tunnel start and the browser launch. I remember this being necessary but can find NO documentation of this.
That said, ember-cli-browserstack certainly works in matrix builds, it has been used that way since the beginning.

There is some documentation over here: https://www.browserstack.com/local-testing/automate#multiple-local-testing-connections
It can currently work with matrix builds most of time because all browser launch quickly. But let's imagine that some browser takes a bit of time to launch in let's say 5 browser matrix build.
By the time fifth launches and start loading the tests index.html file it can fail in mid request simply because Browserstack routed traffic to wrong local instance, because it had multiple instances with same identifier.
It might be less of problem in the context for testing Ember apps but I just recently faced this when I built a local mock testing server running on express.js and I had random failures because traffic was not routed from Browserstack to correct mock testing server instance because they all had same local identifier.

My guess is that it currently works on CI systems that have a build number env var that is unique per matrix run.

Yes, in that case it would also work automagically.

Note: Not using the --local-identifier flag (for each Local Testing connection) will cut off the existing connection and create a new one.

As said in https://www.browserstack.com/local-testing/automate#multiple-local-testing-connections - we should either have safe default by using v4 uuid or then ability to modify the value via something like process.env.BROWSERTACK_LOCAL_IDENTIFIER as I proposed above.
Which way would you prefer? I can open a PR for this.

I think the env var makes sense, it is provided by default by many CI
On Mon, Oct 21, 2019 at 2:08 PM Raido Kuli<EMAIL_ADDRESS>wrote:

Note: Not using the --local-identifier flag (for each Local Testing
connection) will cut off the existing connection and create a new one.
As said in
https://www.browserstack.com/local-testing/automate#multiple-local-testing-connections

we should either have safe default by using v4 uuid or then ability to
modify the value via something like
process.env.BROWSERTACK_LOCAL_IDENTIFIER as I proposed above.

Which way would you prefer? I can open a PR for this.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kategengler/ember-cli-browserstack/issues/10?email_source=notifications&email_token=AADMOOW5SSJYSHLEAUTIV3DQPXVYJA5CNFSM4IUHEFFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB3IDCQ#issuecomment-544637322,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADMOOQ442AVDEYDSTJRI7DQPXVYJANCNFSM4IUHEFFA
.

I opened a PR to address this using env variable.


Bug: Stdout log flushing frequency and partial messages

The current standard out logger isn't flushing after each message. The main issue with this is that when messages are logged only part of the message appears:
Example:
2019-12-11 10:58:41 INFO UDP:  {
  "from": "<IP_ADDRESS>:36616",
  "answer": "<IP_ADDRESS>",
  "server": "hipio-694d565c7-44cgm",
  "question": "<IP_ADDRESS>.hip.io."
}
2019-12-11 10:58:42 INFO UDP:  {
  "from": "<IP_ADDRESS>:39071",
  "answer": "<IP_ADDRESS>",

Flushes seem to happen based on the amount of data and doesn't appear to be time based. This may be partially solved by keeping logs single line as proposed in #7.

Just to verify some assumptions:

Is this exhibiting on the latest release? (0.4.0)
Does output flush completely when the program exits?

Is this exhibiting on the latest release? (0.4.0)

Yup, 0.4.0 is also the first release to do logging to stdout.

Does output flush completely when the program exits?

Nope.
I made a script to help reproduce the problem:
#!/bin/bash

cat > Dockerfile << EOL

FROM centos:8

ADD https://github.com/elastic/hipio/releases/download/v0.4.0/hipio /usr/local/bin/hipio
RUN yum install -y bind-utils && \
    chmod +x /usr/local/bin/hipio

ENTRYPOINT [ "/usr/local/bin/hipio" ]
EOL

docker rm -f hipio || true
docker build -t hipio .
docker run --name hipio --rm -d hipio hipio.local --soa-email test.test
docker logs -f hipio

After starting it up then running:
while true; do docker exec -ti hipio host <IP_ADDRESS>.hipio.local <IP_ADDRESS>; done

While this is running you will see batches of results coming through every so often with the latest log line being incomplete. I have been able to reproduce this running with Docker for Mac, Docker directly running on a Linux machine and also when running via a Docker container in Kubernetes.
One thing that is very odd which I found out while trying to reproduce this is. Is that it does appear to behave correctly when being directly run. If I start a container with docker run --name hipio --entrypoint=/bin/bash --rm -ti hipio and then start it directly interactively the logs flush after each entry. So it seems like the behavior might be different if the session is interactive.


Restarting EC2 Instance Stops Collectd From Sending to CloudWatch?

Hi,
I have somewhat of an urgent and baffling problem in that my collectd and CloudWatch plugin installation will push metrics fine, until the box is restarted. Then, its CloudWatch alarms go into Insufficient Data due to the following error in the collectd logs:
[2016-11-24 13:17:53] [AmazonCloudWatchPlugin][cloudwatch.modules.client.putclient] Request details: 'Action=PutMetricData&MetricData.member.1.Dimensions.member.1.Name=Host&MetricData.member.1.Dimensions.member.1.Value=i-4414404a&MetricData.member.1.Dimensions.member.2.Name=PluginInstance&MetricData.member.1.Dimensions.member.2.Value=NONE&MetricData.member.1.MetricName=collectd.gauge.Tomcat&MetricData.member.1.StatisticValues.Maximum=1.0&MetricData.member.1.StatisticValues.Minimum=1.0&MetricData.member.1.StatisticValues.SampleCount=6&MetricData.member.1.StatisticValues.Sum=6.0&MetricData.member.1.Timestamp=20161124T131653Z&MetricData.member.2.Dimensions.member.1.Name=Host&MetricData.member.2.Dimensions.member.1.Value=i-4414404a&MetricData.member.2.Dimensions.member.2.Name=PluginInstance&MetricData.member.2.Dimensions.member.2.Value=NONE&MetricData.member.2.MetricName=load.load&MetricData.member.2.StatisticValues.Maximum=0.25&MetricData.member.2.StatisticValues.Minimum=0.06&MetricData.member.2.StatisticValues.SampleCount=18&Metric
What could be causing this issue to surface only after reboots? As far as I can tell, all boxes have identical networking, and it can affect any EC2 after I reboot it from the console. Is there maybe some CloudWatch plugin service that does not get started on boot? The collectd service is running fine and I've tried restarting it multiple times.
Thank you.

I've just realized even restarting the collectd service does it. I tried to enable debugging logging and managed to stop another box from pushing just from doing a sudo service collectd restart. The only extra info is that it seems to be trying to push what I expect:
[2016-11-24 13:56:13] [AmazonCloudWatchPlugin][cloudwatch.modules.flusher] [debug] flushing metrics collectd--gauge-Tomcat[5] load--load-[18] df-root-percent_bytes-used[6]


sc is looking for a new maintainer

Since I don't have the time to maintain GitHub projects any more, sc could be transferred to a new owner which continues maintenance.

@n-t-roff I'd be very pleased to take over this role.
We have included sc as part of http://github.com/berryfarm/berrymuch, a CLI software suite for BlackBerry 10, so we want sc to live !

@ other users: Are there any objections?

Indeed, I'm not a C coding expert, so if Maintainer means main C developper of the project, my profile don't match this requirement :-)

Yes, expert C and curses knowledge and very good knowledge of make, Unix, utf8, wide chars, nroff and manpage macros would be required.
Nevertheless quality pull requests are always possible and applied short-term. So you need not to worry regarding the BlackBerry software suite integration.

just found this project... I used vc on Unix Sys V r2.1 back in the (19)80s at Bell Labs... I don't have a ton of time, but assuming it's mostly mature I'm willing to assist maintaining it.

You really had worked at bell labs... sounds interesting.
I have invited you as contributor, normally you should have full access rights to the repository with it.

ya ..  I worked there  1986-2002 .. they finally figured out that the 1982
Divestiture ruling was going to bankrupt them :P
if you're interested in the history...
https://www.historyfactory.com/insights/this-month-in-business-history-bell-system/
in 2007 Stephen Colbert did a bit explaining how Cingular became AT&T
Wireless after AT&T's breakup and subsequent mega-mergers. pretty funny if
you can find it
On Tue, Jul 26, 2022 at 9:01 AM n-t-roff @.***> wrote:

You really had worked at bell labs... sounds interesting.
I have invited you as contributor, normally you should have full access
rights to the repository with it.
—
Reply to this email directly, view it on GitHub
https://github.com/n-t-roff/sc/issues/18#issuecomment-1195599045, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADVJRKCCBLHXFFEPG3OBI3TVV74THANCNFSM4KYV3MFQ
.
You are receiving this because you commented.Message ID:
@.***>

Interesting indeed. Thank you for the information.

greets .. this is an attempt to fix issue  #6 (thought I'd start w/ the
easy one)
I don't want to screw things up w/ my first commit :P
Do you want to try this patch and see how it fairs? (I only tested on Mac
OSX) .. I can test on several flavors of Linux when I get home (currently
on a business trip)
my test:
./configure;make # verify still works in current dir
make clobber; # new target, destroy all built files
mkdir ../sc_build;cd ../sc_build; ../sc/configure; make
make clobber;
if you think it looks ok, I'll commit/push/close issue
On Wed, Jul 27, 2022 at 8:11 AM n-t-roff @.***> wrote:

Interesting indeed. Thank you for the information.
—
Reply to this email directly, view it on GitHub
https://github.com/n-t-roff/sc/issues/18#issuecomment-1196813724, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADVJRKHW5ALIRCA6FXGJ3PTVWE7PVANCNFSM4KYV3MFQ
.
You are receiving this because you commented.Message ID:
@.***>

oops .. I didn't realize I've been replying to this thread the whole time. sorry about that

Thank you. I did extensive tests on the BSDs and Solaris in the past. I'm not quite sure but I believe the build tool chain had completely been POSIX. The patch looks a little bit GNU dependent. Hopefully we don't break compatibility with systems which have POSIX only tools (e.g. some BSDs). Maybe you could try to commit it and rollback if there are complaints from non-GNU systems.
We should continue the discussion in #6.


Resolved Trailing whitespace detected #285

Description
Resolved the trailing whitespace issue. Fixed all the files with trailing whitespace problem.
bug fixes #285
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Database change that requires a migration
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Local manual testing via "./manage runserver"
[x] Local Python Unit Testing via "./manage test"
[x] Local Cypress E2E Testing
[ ] Tested on virtual box
[ ] Tested on server

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

We've noticed that your text editor has made a number of changes to the files - can you please revert those changes back.
Thank you for your help so far.


Bug: iOS13 ASWebAuthenticationSession, no provided presentation context

Solution
Your previous commit at d7779b387fa11baa738c6a3d7f90d2ec21abcd6e is actually working for me now when I updated my local podfile with your reverted changes. Perhaps a test has to be updated. Thanks for the quality work!
Issue
Hi @shreyasthiagaraj , I tried to integrate this SDK into iOS13 + SwiftUI, but am receiving the following error when I call Blockstack.shared.signIn. I see that you've attempted a fix at commit d7779b387fa11baa738c6a3d7f90d2ec21abcd6e
Error
Error Domain=com.apple.AuthenticationServices.WebAuthenticationSession Code=2 "Cannot start ASWebAuthenticationSession without providing presentation context. Set presentationContextProvider before calling -start." UserInfo={NSDebugDescription=Cannot start ASWebAuthenticationSession without providing presentation context. Set presentationContextProvider before calling -start.}
Steps to reproduce

Create a new SwiftUI project
Copy the Blockstack.shared.signIn snippet into a Button.
Build and click button.

This is my first foray into iOS development, so I'm unable to test a UIKit implementation in iOS13.
Code snippet
import Blockstack
import SwiftUI

struct ContentView: View {
    var body: some View {
        NavigationView {
            VStack {
                Text("Hello World")
                Button("Login with Blockstack") {
                    Blockstack.shared.signIn(
                        redirectURI: URL(string:"https://boto-staging.netlify.com/redirect/")!,
                        appDomain: URL(string: "https://boto-staging.netlify.com")!,
                        manifestURI: nil,
                        scopes: [.storeWrite, .publishData]) { authResult in
                            switch authResult {
                            case .success(let userData):
                                print("sign in success")
                                print("userData=\(userData)")
                            case .cancelled:
                                print("sign in cancelled")
                            case .failed(let error):
                                print("sign in failed")
                                print(error!)
                            }
                    }
                }
            }
            .navigationBarTitle("Home")
        }

    }
}

struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}

Resources or solutions I've found/tried

https://www.reddit.com/r/iOSProgramming/comments/cay92l/update_aswebauthenticationsession_and_swiftui/

Although resolved on my end, I'll leave this issue open for you to comment and close as necessary.

@dangdennis Did you use the fix I reverted in d7779b3?

I did use that fix. Although I am noticing that I can get into an error state when the auth is unable to “get information about the app.”
Auth response returns null on my redirect page.
Not sure what causes that yet.

I did use that fix. Although I am noticing that I can get into an error state when the auth is unable to “get information about the app.”
Auth response returns null on my redirect page.
Not sure what causes that yet.

@dangdennis Got it. Looking to see if I can reproduce that issue.


I can't get into Ubuntu, please help

Please paste the output of the following command here: sudo edit-chroot -all
name: precise
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/precise...
crouton: version unknown
release: unknown
architecture: unknown
xmethod: xorg
targets: unity
host: version 7834.60.0 (Official Build) stable-channel quawks
kernel: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/precise...

Please describe your issue: I just got Ubuntu 12.04 LTS and tried upgrading to 14.04 LTS. It said that the upgrade finished but there were errors or something and I rebooted my chromebook and came back to the crosh shell. I put in sudo startunity and it says this:
chronos@localhost / $ sudo startunity
Entering /mnt/stateful_partition/crouton/chroots/precise...
Creating hwaudio group with GID 18...
Changing cdrom GID from 24 to 19...
Changing floppy GID from 25 to 11...
Changing plugdev GID from 46 to 85...
Changing uucp GID from 10 to 14...
Changing video GID from 44 to 27...
Moving sudo GID from 27 to 28...
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
X.Org X Server 1.15.1
Release Date: 2014-04-13
X Protocol Version 11, Revision 0
Build Operating System: Linux 3.2.0-76-generic x86_64 Ubuntu
Current Operating System: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64
Kernel command line: cros_secure console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 2506752 verity payload=PARTUUID=1bafbe4d-a93e-3d41-be36-ea940fdb11e9/PARTNROFF=1 hashtree=PARTUUID=1bafbe4d-a93e-3d41-be36-ea940fdb11e9/PARTNROFF=1 hashstart=2506752 alg=sha1 root_hexdigest=0f5340f74cd923b0d348939e445ed16762560183 salt=a2b9f363ce95098ff7c8c444935eb0548b4a07cd3c08fe9bd826c96d21a6fce1" noinitrd vt.global_cursor_default=0 kern_guid=1bafbe4d-a93e-3d41-be36-ea940fdb11e9 add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic
Build Date: 12 February 2015  02:49:29PM
xorg-server 2:1.15.1-0ubuntu2.7 (For technical support please see http://www.ubuntu.com/support)
Current version of pixman: 0.30.2
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/Xorg.crouton.1.log", Time: Thu Mar 24 11:59:08 2016
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
Initializing built-in extension XC-MISC
Initializing built-in extension SECURITY
Initializing built-in extension XINERAMA
Initializing built-in extension XFIXES
Initializing built-in extension RENDER
Initializing built-in extension RANDR
Initializing built-in extension COMPOSITE
Initializing built-in extension DAMAGE
Initializing built-in extension MIT-SCREEN-SAVER
Initializing built-in extension DOUBLE-BUFFER
Initializing built-in extension RECORD
Initializing built-in extension DPMS
Initializing built-in extension Present
Initializing built-in extension DRI3
Initializing built-in extension X-Resource
Initializing built-in extension XVideo
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension XFree86-VidModeExtension
Initializing built-in extension XFree86-DGA
Initializing built-in extension XFree86-DRI
Initializing built-in extension DRI2
Loading extension GLX
xf86EnableIOPorts: failed to set IOPL for I/O (Operation not permitted)
crouton: version 1-20160206234800~master:daa872a0
release: precise
architecture: amd64
xmethod: xorg
targets: unity
host: version 7834.60.0 (Official Build) stable-channel quawks
kernel: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
No Chromium OS X server is available.
Running exit commands...
/usr/bin/xinit: connection to X server lost
waiting for X server to shut down Hangup
(EE) Server terminated successfully (0). Closing log file.
Unmounting /mnt/stateful_partition/crouton/chroots/precise...
Then, I updated my chroot:
chronos@localhost / $ sudo sh -e ~/Downloads/crouton -n precise -u
Downloading latest crouton installer...
######################################################################## 100.0%
/usr/local/chroots/precise already exists; updating it...
Preparing chroot environment...
Installing brightness into the chroot...
Installing croutonpowerd into the chroot...
Installing croutonversion into the chroot...
Installing host-dbus into the chroot...
Installing host-x11 into the chroot...
Installing volume into the chroot...
Installing pulseaudio-default.pa into the chroot...
Installing croutoncycle into the chroot...
Installing croutontriggerd into the chroot...
Installing croutonxinitrc-wrapper into the chroot...
Installing setres into the chroot...
Installing xinit into the chroot...
Installing xbindkeysrc.scm into the chroot...
Installing xorg-intel-sna.conf into the chroot...
Installing xserverrc into the chroot...
Installing xserverrc-xorg into the chroot...
Installing xserverrc-local.example into the chroot...
Installing startunity into the host...
Installing crouton-noroot into the chroot...
Installing startunity into the chroot...
Installing gnome-session-wrapper into the chroot...
Installing crouton-unity-autostart into the chroot...
Installing unity-autostart.desktop into the chroot...
Installing unity-profiled into the chroot...
Installing enter-chroot into the host...
Installing delete-chroot into the host...
Installing edit-chroot into the host...
Installing mount-chroot into the host...
Installing unmount-chroot into the host...
Installing crash_reporter_wrapper into the host...
dpkg: dependency problems prevent configuration of printer-driver-gutenprint:
printer-driver-gutenprint depends on libgutenprint2 (>= 5.2.10~pre2); however:
Version of libgutenprint2 on system is 5.2.8~pre1-0ubuntu2.1.
dpkg: error processing package printer-driver-gutenprint (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
printer-driver-gutenprint
Failed to complete chroot setup.
Unmounting /mnt/stateful_partition/crouton/chroots/precise...
I tried sudo startunity again:
chronos@localhost / $ sudo startunity
Entering /mnt/stateful_partition/crouton/chroots/precise...
A chroot setup script still exists inside the chroot.
The chroot may not be fully set up.
Would you like to finish the setup? [Y/n/d] y
Preparing chroot environment...
dpkg: dependency problems prevent configuration of printer-driver-gutenprint:
printer-driver-gutenprint depends on libgutenprint2 (>= 5.2.10~pre2); however:
Version of libgutenprint2 on system is 5.2.8~pre1-0ubuntu2.1.
dpkg: error processing package printer-driver-gutenprint (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
printer-driver-gutenprint
Failed to complete chroot setup.
The chroot setup script may be broken. Your chroot is not fully configured.
Removing the chroot setup script. You may want to update your chroot again.
_XSERVTransmkdir: Owner of /tmp/.X11-unix should be set to root
X.Org X Server 1.15.1
Release Date: 2014-04-13
X Protocol Version 11, Revision 0
Build Operating System: Linux 3.2.0-76-generic x86_64 Ubuntu
Current Operating System: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64
Kernel command line: cros_secure console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 2506752 verity payload=PARTUUID=1bafbe4d-a93e-3d41-be36-ea940fdb11e9/PARTNROFF=1 hashtree=PARTUUID=1bafbe4d-a93e-3d41-be36-ea940fdb11e9/PARTNROFF=1 hashstart=2506752 alg=sha1 root_hexdigest=0f5340f74cd923b0d348939e445ed16762560183 salt=a2b9f363ce95098ff7c8c444935eb0548b4a07cd3c08fe9bd826c96d21a6fce1" noinitrd vt.global_cursor_default=0 kern_guid=1bafbe4d-a93e-3d41-be36-ea940fdb11e9 add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic
Build Date: 12 February 2015  02:49:29PM
xorg-server 2:1.15.1-0ubuntu2.7 (For technical support please see http://www.ubuntu.com/support)
Current version of pixman: 0.30.2
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/Xorg.crouton.1.log", Time: Thu Mar 24 12:02:48 2016
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
Initializing built-in extension Generic Event Extension
Initializing built-in extension SHAPE
Initializing built-in extension MIT-SHM
Initializing built-in extension XInputExtension
Initializing built-in extension XTEST
Initializing built-in extension BIG-REQUESTS
Initializing built-in extension SYNC
Initializing built-in extension XKEYBOARD
Initializing built-in extension XC-MISC
Initializing built-in extension SECURITY
Initializing built-in extension XINERAMA
Initializing built-in extension XFIXES
Initializing built-in extension RENDER
Initializing built-in extension RANDR
Initializing built-in extension COMPOSITE
Initializing built-in extension DAMAGE
Initializing built-in extension MIT-SCREEN-SAVER
Initializing built-in extension DOUBLE-BUFFER
Initializing built-in extension RECORD
Initializing built-in extension DPMS
Initializing built-in extension Present
Initializing built-in extension DRI3
Initializing built-in extension X-Resource
Initializing built-in extension XVideo
Initializing built-in extension XVideo-MotionCompensation
Initializing built-in extension SELinux
Initializing built-in extension XFree86-VidModeExtension
Initializing built-in extension XFree86-DGA
Initializing built-in extension XFree86-DRI
Initializing built-in extension DRI2
Loading extension GLX
xf86EnableIOPorts: failed to set IOPL for I/O (Operation not permitted)
crouton: version unknown
release: unknown
architecture: unknown
xmethod: xorg
targets: unity
host: version 7834.60.0 (Official Build) stable-channel quawks
kernel: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
No Chromium OS X server is available.
Running exit commands...
/usr/bin/xinit: connection to X server lost
waiting for X server to shut down Hangup
(EE) Server terminated successfully (0). Closing log file.
Unmounting /mnt/stateful_partition/crouton/chroots/precise...
I'm frustrated. Anyone help?

@TheGlamourGroup -

Failed to complete chroot setup.

There are way too many 'unknown's in your 'croutonversion' output.
It might be best if you entered it and let the setup script finish, if it's still there hopefully, by doing this:
sudo enter-chroot -u root -n precise -xx
-DennisL

I actually ended up deleting Ubuntu and getting out of developer mode, and restarting the whole again. but what do the 'unknown's indicate, do I need to update crouton? I'm reusing the same copy and I got it kinda a while ago.

@TheGlamourGroup -

I actually ended up deleting Ubuntu and getting out of developer mode, and restarting the whole thing again.

If you got out of Developer mode, you're crouton chroot will have been removed along with all of the user data.

but what do the 'unknown's indicate, do I need to update crouton?

It means the crouton installation didn't finish and a setup script is was still there.

I'm reusing the same copy and I got it kinda a while ago.

I don't know what you mean by that.
You'll need to get a fresh copy of 'crouton' and start again - I wouldn't use your old chroot even if you have it backed up.
-DennsiL

@DennisLfromGA Yeah I backed up my copy of crouton and reuse that every time I reinstall Ubuntu. I'll try a new copy if I run into anymore problems

@TheGlamourGroup - Why don't you try a new copy before you run into anymore problems???

@DennisLfromGA The update was already going before I got a new copy and it finished now but i'm back to the same error again, but this time the unknowns are replaced:
chronos@localhost / $ sudo edit-chroot -all
name: precise
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/precise...
crouton: version 1-20160206234800~master:daa872a0
release: precise
architecture: amd64
xmethod: xorg
targets: unity
host: version 7834.60.0 (Official Build) stable-channel quawks
kernel: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/precise...
Ubuntu said to restart the computer to finish updating but I can't even get into it

@TheGlamourGroup,

...but this time the unknowns are replaced

Believe it or not, that's great progress.

Ubuntu said to restart the computer to finish updating but I can't even get into it

Now just reboot, open a crosh shell, and let it finish the setup by entering:
sudo enter-chroot -u root -n precise -xx
After that completes successfully, exit the chroot and try:
sudo startunity

A word of advice: The 'precise' release is years old, I would recommend using the newer 'trusty' LTS release.
-DennisL

@DennisLfromGA When I enter that it says:
chronos@localhost / $ sudo enter-chroot -u root -n precise -xx
chroot: failed to run command '/prepare.sh': No such file or directory
Failed to complete chroot setup.
Unmounting /mnt/stateful_partition/crouton/chroots/precise...

@TheGlamourGroup,

Ubuntu said to restart the computer to finish updating but I can't even get into it

Okay, from the above I thought it hadn't finished since you said you couldn't get into it.
Maybe it did finish and all is well; just try this and see if you can get into it: sudo enter-chroot -n precise
If that works and you get a shell terminal session, then exit and try: sudo startunity -n precise
-DennisL

@DennisLfromGA When I put in the sudo enter-chroot -n precise, it says this:
chronos@localhost / $ sudo enter-chroot -n precise
Entering /mnt/stateful_partition/crouton/chroots/precise...
A chroot setup script still exists inside the chroot.
The chroot may not be fully set up.
Would you like to finish the setup? [Y/n/d] y
Preparing chroot environment...
dpkg: dependency problems prevent configuration of printer-driver-gutenprint:
printer-driver-gutenprint depends on libgutenprint2 (>= 5.2.10~pre2); however:
Version of libgutenprint2 on system is 5.2.8~pre1-0ubuntu2.1.
dpkg: error processing package printer-driver-gutenprint (--configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
printer-driver-gutenprint
Failed to complete chroot setup.
The chroot setup script may be broken. Your chroot is not fully configured.
Removing the chroot setup script. You may want to update your chroot again.

@TheGlamourGroup,

You may want to update your chroot again.

Try that first: sudo sh ~/Downloads/crouton -n precise -u`
If that doesn't work, get back into your chroot: sudo enter-chroot -n precise
If it wants to finish the setup script again, let it, then try this in the chroot:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get -f install
See if that completes successfully, if so, try starting 'unity' again.
-Dennis

@DennisLfromGA Ok it says (precise)theglamourgroup@localhost:~$ like how it would in the Ubuntu terminal, I just never noticed it. I'm updating using those commands now

@TheGlamourGroup -
It looks like you've regressed and your back to square one with all the 'unknown's.
I had asked you to "Try that first: sudo sh ~/Downloads/crouton -n precise -u`"
That might have 'fixed' it but we'll never know.
At this point I would 'punt' - just remove the chroot and install the 'trusty' release this time IMHO.
-DennisL

@DennisLfromGA How do I install the trusty release? Whenever I install Ubuntu it always takes me to 12.04 LTS.

@TheGlamourGroup - Please familiarize yourself with the crouton README, the crouton WIKI, and other sources first - it will save you and others unnecessary problems and false starts.

This may be good news, but I did the sudo sh ~/Downloads/crouton -n precise -u command and it worked. I also noticed the unknowns are replaced again:
chronos@localhost / $ sudo edit-chroot -all
name: precise
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/precise...
crouton: version 1-20160206234800~master:daa872a0
release: trusty
architecture: amd64
xmethod: xorg
targets: unity
host: version 7834.60.0 (Official Build) stable-channel quawks
kernel: Linux localhost 3.10.18 #1 SMP Thu Mar 10 05:45:38 PST 2016 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/precise...


Changes for FosUserBundle

Coverage decreased (-18.89%) to 79.03% when pulling 8de957f9dab035c2d15963a69ab9cb53c9e7ad63 on garwat82:fosuser into ece7a16820bc475c7735eac61c451e834c8dae8e on aimeos:master.


🐛 Available Work Types page throws an error

Summary
ref convo
In prep for the mobius demo we've discovered the https://mobius-staging.staging.hykuup.com/admin/work_types/edit page crashes.
The error traces back to a view file (/app/samvera/app/views/admin/work_types/edit.html.erb)that does NOT exist in the repo, and therefore shouldn't be on the server (but it is) :thinking_face:
We tried to manually delete it off the server which appeared to nearly take down the site entirely - we weren't able to login (authenticity token error) which seems totally unrelated. Redeploying resolved this but now we're back in the state where this ghost file persists.
Acceptance Criteria

[x] Visiting the work types edit page should not result in a crash

Screenshots or Video
Testing Instructions
Notes

Kiah rebuilt base:

https://github.com/samvera/hyku/actions/runs/8729422098

Rebuilding the base Hyku image was the right call. The issue was, the HykuUP Knapsack image was cached in Rancher, so the updated image built on top of the new base didn't actually deploy. This is because the Docker image tag was the same.
We deployed a new image with a different sha that was created after merging a PR. This resolved this issue.


anyone ran into OOM with 7B?

I have a A100 40GB GPU and ran this command:
python finetune.py \
    --base_model='decapoda-research/llama-7b-hf' \
    --num_epochs=10 \
    --cutoff_len=512 \
    --group_by_length \
    --output_dir='./alpaca-lora-saved-model' \
    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
    --lora_r=16 \
    --micro_batch_size=8

But I ran into OOM when it was calculating attentions:
...
│   250 │   │   │   attn_weights = torch.max(attn_weights, torch.tensor(torch.finfo(attn_weights   │
│   251 │   │                                                                                │
│   252 │   │   # upcast attention to fp32                                                         │
│ ❱ 253 │   │   attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).   │
│   254 │   │   attn_output = torch.matmul(attn_weights, value_states)                             │
│   255 │   │                                                                                │
│   256 │   │   if attn_output.size() != (bsz, self.num_heads, q_len, self.head_dim):              │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
OutOfMemoryError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 39.41 GiB total capacity; 37.41 GiB already allocated; 64.50 MiB free; 38.15 GiB
reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory
Management and PYTORCH_CUDA_ALLOC_CONF

Any thoughts? Thanks.

Did you modify something in the finetune.py? I've trained 30B with single A100 40G before and it took only 35G VRAM.

I get the same error when I run the code that had previously worked without issues a week ago. I have attempted to install older versions of libraries, but this has not resolved the error.

@GooDRomka I decided to recreate a new VM and now it actually works fine. I grabbed dependencies version on my VM (a couple of them do not have __version__ so they are not included) in case you want to compare:
{
    "accelerate": "0.18.0",
    "appdirs": "1.4.4",
    "loralib": NA,
    "bitsandbytes": NA,
    "black": "23.3.0",
    "datasets": "2.11.0",
    "fire": "0.5.0",
    "peft": "0.3.0.dev0",
    "transformers": "4.29.0.dev0",
    "sentencepiece": "0.1.98",
    "gradio": "3.25.0",
}


README.md installation on OSX wrong

If you check the points you'll see the following one:

Download qt5.4.2 - https://download.qt.io/official_releases/qt/5.4/5.4.2/qt-opensource-mac-x64-clang-5.4.2.dmg

This link is dead, can't install it.

You can download the 5.4.2 version from archive folder
https://download.qt.io/archive/qt/5.4/5.4.2/

@justinvforvendetta  I have a fix for this

working download url
update qt version to 5.5 to match the version in build.sh
updates to readme qt install PATH
removes some trailing white spaces

Can you make me a contributor and I'll push the branch?

I'll post a PR shortly with updated build instructions.

@ekryski is everything else ok with this - i.e. qt5.5? it seems the qt5.5 download installer gets stuck for some people. Is this a blocker?


Artifact/dependency from docs is 404

Describe the bug
I was checking out your docs at https://docs.talsec.app/freerasp/integration/cordova#install-the-plugin and came across this instruction:
Set Talsec's Artifact Registry in your project's build.gradle (or settings.gradle if you are using settings repositories):

[build.gradle (:project)]
Copy
repositories {
    google()
    mavenCentral()
    maven { url "https://jitpack.io" }
    maven { url "https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp" }
}

Howeve it appears that this link is dead, it leads to a 404 page:
https://europe-west3-maven.pkg.dev/talsec-artifact-repository/freerasp
To Reproduce
If applicable, steps to reproduce the behavior.
Expected behavior
Either update or remove the link
Screenshots
If applicable, add screenshots to help explain your problem.
Please complete the following information:

Device: [e.g. iPhone6]
OS version: [e.g. iOS8.1]
Version of freeRASP: [e.g. 22]

Additional context
Add any other context about the problem here.

Hello @mchl18,
A dependency is hosted on the link, see further:
dependencies {     // freeRASP SDK       implementation 'com.aheaditec.talsec.security:TalsecSecurity-Community:9.1.0' }
Could you try if you can download it successfully?
Kind regards,
Talsec team


For names with a space in the middle, the second part gets removed

For first names that have a space in the middle (e.g. 'Yeon Soo'), name throughout the app incorrectly appears as just 'Yeon'.

^ @ChenJesse can this issue be closed?

I updated and my name is shown as Ji Kim

Looks like my name is stored as Ji Hun Kim in the DB but it shows up as Ji Kim

lemme look into it

Yea that bug is appearing on the TestFlight version right? It's fixed on the simulator but not on the TestFlight for me, so maybe the TestFlight isn't completely up to date?

Yep on the TestFlight

Yea pretty sure it's just TestFlight not being completely up to date, this is my simulator of most recent master branch


SUBGRAPH: Add the implications/anti-implications between 4564 and the other equations as conjecture statements.

Note that 4564 is dual to 4513, so one should be able to adapt many of the existing implications.

Closing due to obsolete workflow now that more automated tools are available.


Code for evaluation of Euros Models

Can you add the code for reproducing the main results in the paper for various math and coding datasets, along with their prompts and the data splits used?

Thanks for your interest! We are working on it right now and we will release the evaluation code soon.

Thanks @cgq15, in the meantime could you let me know the generation configs used for different task types? From what I have been able to reproduce the numbers are not consistent with Table 3 of your paper (see below), and I think it can be attributed to generation configs such as temperature, top_p, top_k, and do_sample as I am using the same prompts as listed in the paper.
Reproduction Study:
Dataset: HumanEval, Reported Performance: 55.5, Performance Obtained: 47.56 (temp=0.2, top_p=0.9, top_k=50)
Dataset: MATH, Reported Performance: 32.6, Performance Obtained: 29.6 (PoT with above parameters)
Note: I have loaded the model with 8-bit quantization.

Hi, we are releasing the eval code today. So please stay tuned.
For hyperparameters, we set temperature=0, i.e. greedy decoding for all coding and math tasks. Also, we test the models with float16 weights.

Hi @archiki,
we have released the eval code. Enjoy!

Thanks a lot! @lifan-yuan can you clarify if the performance reported on MBPP and HumanEval are on the regular test sets or the EvalPlus suites? TIA!


KConfig?

The standard convention in ESP-IDF libraries is to use KConfig / menuconfig / sdkconfig for user configuration of libraries/components. Right now this project uses a couple of makefile flags as well as header defines for configuration of the library for the end-user's device. I would suggest a move to use all header defines but have those defines in turn reference sdkconfig variables.
So in the header for example we'd change this:
#define DISP_SPI_MOSI 23

to
#define DISP_SPI_MOSI CONFIG_DISP_SPI_MOSI

CONFIG_DISP_SPI_MOSI in turn is set in sdkconfig directly or through menuconfig, and will still have a default value. The two layers of defines gives you the flexibility to quickly change it in the library header while testing, which does not recompile the whole sdk.
The advantage for the user is this similar to other libraries as well as all the IDF components themselves, but also that they do not necessarily have to vendor the library, they can keep it as a git submodule without worrying about the dirty flags.
I very recently implemented such a scheme for the loboris driver and would be happy to submit a PR for this library as well.

I very recently implemented such a scheme for the loboris driver and would be happy to submit a PR for this library as well.

It sounds very well. Please send a PR!

I'm pretty close to having a PR ready, should be in the next couple days, all that is left are instructions in the README. WIP is on this branch: https://github.com/jeremyjh/esp32_ili9341/tree/kconfig

Awesome, thank you for the feedback.
Do you know about the new Cmake based build system of ESP? If so have you already make it work this project?

Yes Cmake support is in that branch too.
On Thu, Oct 24, 2019 at 7:45 AM Gabor Kiss-Vamosi<EMAIL_ADDRESS>wrote:

Awesome, thank you for the feedback.
Do you know about the new Cmake based build system of ESP? If so have you
already make it work this project?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/littlevgl/esp32_ili9341/issues/20?email_source=notifications&email_token=AAAWDDSTTFT6ZPM3BGNPR2TQQGDFLA5CNFSM4I4S5UI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECEXQHI#issuecomment-545880093,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAWDDWI7IQTW3W5A46WIP3QQGDFLANCNFSM4I4S5UIQ
.

Cool :sunglasses:


Add Venta Blacl by Leon Riskin

Song Name
Venta Blacl
Artist Name
Leon Riskin
Source
Youtube
Youtube Link
https://youtu.be/yiHfvsUsj5c?si=LellrrdpFsWWxV7m
Direct File Link
No response
Song ID
1
Start Offset [ms]
No response
Extra
Created from Auto Nong.
Created for level: 107216500

reject


.github: Rename project/ci-force to ci/flake

Following discussion in the community meeting, we decided to rename the project/ci-force label to ci/flake. We need to rename it in MLH and the issue template.

Marking as ready given Aditi reviewed and there are no tests covering it.

Marking for backports since MLH uses the config file from the branch where the flake happened.


Keyboard layout changes are not detected on Linux

VSCode Version: Code - Insiders 1.11.0-insider (6b9548065ad3e31fca4f0e91cf882505fe8c30cf, 2017-03-30T06:18:34.366Z)
OS Version: Linux x64 3.13.0-108-generic
Extensions: none

Found verifying #1549
Steps to Reproduce on Linux under Parallels with a US keyboard (did not reproduce on Mac):

Start with US layout
Navigate in code to populate navigation history
Navigate back / forward with keyboard shortcut (Ctrl + Alt + Minus, Ctrl + Shift + Minus)
Switch to German layout
Navigate back / forward now works with both the German location of Minus (expected) and the US location of Minus (unexpected)
Reload window
Now navigation only works with the German location of Minus (expected)

It seems like the layout switch doesn't clear out the old binding.

Yes, keyboard layout changes are not detected on Linux. The workaround is to restart vscode

restarting VS code dos not make it loar right keybord for shortkut

@nicolaslussier Have you tried the workaround suggested at https://github.com/Microsoft/vscode/issues/24166#issuecomment-294002885

I'm having trouble with this, especially because I open VS Code with the French keyboard layout enabled (my OS default), then I switch to US for coding. The Ctrl+Z and Ctrl+W bindings stay on the French keyboard layout which is super frustrating... I haven't found a workaround that helps for now.

@Spone You can reload VS Code F1 > Developer: Reload Window

Yes, but doing so every time I launch VS Code? :rofl:

More possible workarounds:

launch code with a script where you change the keyboard layout right before launching via setxkbmap

(or)

use keyboard layout independent keybindings (code based). e.g. use the following format in your keybindings.json for those keys that annoy you:

{ "key": "cmd+[KeyZ]", "command": "undo", "when": "textInputFocus && !editorReadonly" },

Oh, great! The second workaround did the job! Thanks a lot @alexandrudima that was the main pain point while adopting VS Code :)

I am switching frequently between EN and FR keyboard. This bug makes VSCode barely usable for me on Ubuntu 18.04.

I am using EN and FR keyboards on Ubuntu 18.04. I made a custom alias for the command : setxkbmap fr  that I use when I'm switching keyboard. It works for me.
Thank you @alexdima

Exact same issue, cannot find a way to fix.

@alexdima you mention

use keyboard layout independent keybindings (code based). e.g. use the following format in your keybindings.json for those keys that annoy you:
{ "key": "cmd+[KeyZ]", "command": "undo", "when": "textInputFocus && !editorReadonly" },

Since this is not how the default keybindings are defined, I assume that there are drawbacks to this method, can you tell what they are (or why the default keybindings don't use this ? )
thanks

Same issue here. Nothing changed, except update to Version: 1.53.2

I would just prefer to have all the open workspaces to be open again automatically.

File > exit (or Ctrl + Q) will quit all windows
Then when you start code again (without any parameter), it restores the latest windows by default.

ubuntu 16 and latest vs code still an issue.


replace withTheme HOC with withTheme option.

This PR closes #876
Description
This pull request fixes the issue with @material-ui/styles. The theme prop is undefined in old version of DateTimePickerTabs component.

It is only one place?

when are you planning to release it?


refactor(java): improve oneOf types DX

🧭 What and Why
Currently, all oneOf types are built using a wrapper, even if no wrapper is needed. Also, generics are added for oneOf types too, although it doesn't make sense, especially if the underlying types are different (e.g., different records from each index).
Changes included:

Refactor the oneOf types to become interfaces, with concrete implementation when the oneOf type is a class or enum.
Use wrappers in the case of primitives and collections.
During deserialization, always start with types that have discriminator fields.
Move from jetbrains @NotNullto javax @Nonnull

ticket

There is, in fact, SearchResponse<T> which remains generic, and there's SearchResult, which is the response used exclusively for multi-searches. In that context, generics don't make sense :D


Molecule 5.0 Failing on GitHub Action

Issue Type

Bug report

Molecule and Ansible details
ansible --version && molecule --version
ansible-7.4.0
molecule-5.0.0

Molecule installation method (one of):

pip

Ansible installation method (one of):

pip

Detail any linters or test runners used:
Desired Behavior
Molecule not failing to run.
Actual Behaviour
Since release of molecule 5.0 my github action is failing with the following error:
https://github.com/dgibbs64/ansible-role-linux_admin_packages/actions/runs/4759492512/jobs/8458829447
Traceback (most recent call last):
File "/home/runner/.local/bin/molecule", line 8, in
sys.exit(main())
File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/runner/.local/lib/python3.10/site-packages/molecule/command/test.py", line 113, in test
base.execute_cmdline_scenarios(scenario_name, args, command_args, ansible_args)
File "/home/runner/.local/lib/python3.10/site-packages/molecule/command/base.py", line 94, in execute_cmdline_scenarios
get_configs(args, command_args, ansible_args, glob_str),
File "/home/runner/.local/lib/python3.10/site-packages/molecule/command/base.py", line 185, in get_configs
configs = [
File "/home/runner/.local/lib/python3.10/site-packages/molecule/command/base.py", line 186, in
config.Config(
File "/home/runner/.local/lib/python3.10/site-packages/molecule/config.py", line 64, in call
obj.after_init()
File "/home/runner/.local/lib/python3.10/site-packages/molecule/config.py", line 116, in after_init
self.config = self._reget_config()
File "/home/runner/.local/lib/python3.10/site-packages/molecule/config.py", line 306, in _reget_config
env = util.merge_dicts(os.environ, self.env)
File "/home/runner/.local/lib/python3.10/site-packages/molecule/config.py", line 201, in env
"MOLECULE_INSTANCE_CONFIG": self.driver.instance_config,
File "/usr/lib/python3.10/functools.py", line 981, in get
val = self.func(instance)
File "/home/runner/.local/lib/python3.10/site-packages/molecule/config.py", line 185, in driver
driver = api.drivers(config=self)[driver_name]
File "/home/runner/.local/lib/python3.10/site-packages/molecule/api.py", line 30, in getitem
return self.dict[i]
KeyError: 'docker'

I could have observed the same bug on my GitHub action: https://github.com/kmezynski/ansible-role-ohmyzsh/actions/runs/4756760785/jobs/8452737921. No changes to the role itself were made since last successful build.

PRs are more than welcomed ;)

oops
https://github.com/vitabaks/postgresql_cluster/actions/runs/4760126337

The original reporter had a GitHub Action that ran pip install molecule[docker]. This no longer works in Molecule 5.0.
The same is true for the podman driver. This used to work:
pip install install molecule[podman]==4.0.4

This no longer works in 5.0.0:
pip install install molecule[podman]==5.0.0
WARNING: molecule 5.0.0 does not provide the extra 'podman'

The replacement is documented:
pip install molecule-plugins[podman]

It would be nice if the error message was clearer instead of simply raising KeyError. If you raise MoleculeDriverNotFound (maybe with a link to the Molecule installation docs) it would be easier for users to search with Google.

I was also impacted by this today and here is a proposition to make it clear in the documentation (https://github.com/ansible-community/molecule/pull/3899).

So to confirm the fix is to change molecule[docker] to molecule-plugins in my github action.
It would be good to have some sort of error message when the failure happens


Draft: Implement to WKT for Geometries

Some exploration for ToWKT. It might help with #762, but unsure what that entails. I'm just putting it here for feedback on whether this is a viable approach.

Thanks for the contribution @b4l!
I was originally thinking that ToWKT would wrap the geozero WKT writer.
I think it's beneficial for us to have a custom WKB implementation because WKB use is so widespread in the ecosystem (e.g. GeoParquet) and having a very fast WKB parser and encoder is important.
But I think WKT performance is less critical because if someone really cares about performance why are they storing WKT to begin with 😄? So I'm less sure that a custom WKT parser/encoder is worth the maintenance overhead (when the project is already super large).
That said, I've been hitting some core issues with geozero, such as in https://github.com/georust/geozero/pull/237, where I think geozero is somewhat fundamentally broken in its handling of Z and M dimensions. So in general I've been moving towards a greater decoupling from geozero where possible. E.g. https://github.com/geoarrow/geoarrow-rs/pull/779 laid the groundwork for not using geozero for parsing Flatgeobuf geometries and https://github.com/geoarrow/geoarrow-rs/pull/759 does the same for Shapefile.
Any thoughts on this?

I think decoupling geozero is desirable as it is modeled for rows, whereas the arrow model is columnar

I agree it's desirable, but just a question of what maintenance cost. I think a custom implementation of writing to WKT would probably be significantly easier than a custom parser implementation.

though not for i/o but rather as a display/debug implementation to use within analytical processes/workflows or displaying SQL query results.

I agree having display/debug is very valuable, but why not also use this for I/O if we're implementing a WKT writer anyways?

I just mean that this WKT writer can be used for more than just debugging and display.
So for example we could offer a to_csv method, or something like that, that uses this WKT encoder under the hood.
It would also be good for the bindings to offer a to_wkt method, which mirrors e.g. shapely.to_wkt.

I think a good place to start with this PR is ensuring we have correct writing of scalars to WKT, and leaving writing of arrays for follow ups.
What do you think about like including the tests from geozero's WKT writer to just ensure we have parity for writing scalars?

The latest commit reminded me that the implementation of reading/writing WKT is not actually in geozero; it's in https://github.com/georust/wkt.
The geozero wrapper on top handles the streaming read/write of WKT and the integration into geozero traits. But wkt is usable on its own.
Unless there's some reason why we can't use wkt directly, I'd be strongly in favor of depending on WKT and not rolling our own wkt writer.

I think decoupling geozero is desirable as it is modeled for rows, whereas the arrow model is columnar

To follow up on this, I think it's important to note where geozero does and does not fit in. In particular for export to WKT, it's easy for Arrow to handle columnar strings, so it's easy to use geozero (or wkt directly) for this.
It's harder for parsing to geometries because we need to handle the special builders for geometry data compared to string data.

The main difference to wkt is that this enables to write geometries directly to a string array. With wkt it is geometry > intermediate representation > string > sink. The former could be enabled in wkt, though it would require some substantial refactoring.

In https://github.com/geoarrow/geoarrow-rs/pull/791 yesterday I laid the groundwork for parsing WKT from the wkt crate without going through geozero. It might be marginally faster but mainly it means that it's a simpler implementation since we have full access to all the coordinates upfront rather than needing to handle the stream geozero gives us.


crashes on borken symlinks

This tools is fantastic-- but it seems crash on broken symlinks.  Telling it to ignore 0 byte files doesn't solve it, since the symlinks are not 0 bytes.  Maybe I'm missing something...
www/boi-cms.old/taps_ctrl.sh
Traceback (most recent call last):
File "/usr/local/bin/boto-rsync", line 1150, in
main()
File "/usr/local/bin/boto-rsync", line 544, in main
file_size = os.path.getsize(fullpath)
File "/usr/lib/python2.7/genericpath.py", line 49, in getsize
return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/var/www/boi-cms.old/current'
Scanning for files to transfer...
shells
Muttrc
pam.conf
nanorc
legal
Traceback (most recent call last):
File "/usr/local/bin/boto-rsync", line 1150, in
main()
File "/usr/local/bin/boto-rsync", line 544, in main
file_size = os.path.getsize(fullpath)
File "/usr/lib/python2.7/genericpath.py", line 49, in getsize
return os.stat(filename).st_size
OSError: [Errno 2] No such file or directory: '/etc/blkid.tab'
ckelty@beadle:/etc/backup$ ls -al /etc/blkid.
blkid.conf  blkid.tab
ckelty@beadle:/etc/backup$ ls -al /etc/blkid.tab
lrwxrwxrwx 1 root root 15 2011-07-23 16:47 /etc/blkid.tab -> /dev/.blkid.tab
ckelty@beadle:/etc/backup$ ls -al /var/www/boi-cms.old/current
lrwxrwxrwx 1 capistrano www-data 40 2012-04-12 14:20 /var/www/boi-cms.old/current -> /var/www/boi-cms/releases/20120412212119
ckelty@beadle:/etc/backup$

You can avoid purging dangling symlinks by applying zackse/boto_rsync@55edef2c9d4bf6cba6d0cdf1d82b9c80f4c29257 , which will skip any files that cannot be stat()'ed.


Why setting CPU resource requirements to 0 in cache preparation?

Hi, thanks for the great work!
I'm curious why you choose to set cpu requirement to 0.0 in a bunch of ray tasks/actors during dataset caching, for example:
https://github.com/stanford-crfm/levanter/blob/133dbba64ff08873ed07a5661211db867a471343/src/levanter/data/shard_cache.py#L254
I've been trying to cache a fairly large dataset (C4) on a reasonably large number of CPUs (slightly more then 1K); every time I run the dataset caching script I run into resource exhaustion issues. Setting the cpu requirements to 1 or 2 generally improves things significantly though it can still lead to resource exhaustion due to ray releasing resource associated with tasks waiting on ray.get().
So I'm curious to know what's the rationales behind resource requirement setting and whether this is potentially a bug.

Hi @tjingrant,
Hrm, that's interesting. I'll be honest I didn't really tune these beyond making sure that a TPU v3-32 (8 machines with ~48 cores) could handle The Pile. The main use case is for preprocessing running in the background while training is running on the CPUs that control training, so I didn't really tune for the 1000s of CPUS in the "offline" case.
I guess I thought it was doing basically no work (besides json parsing), so it was fine to set it to 0. (The tasks that actually do preprocessing have their own resource requirements. (This is what _mk_queue_aware_process_task is doing)
I do need to improve this though. I will try to look into this this week.

i'm working on fixing this right now, but given my history with Ray this is likely to take a few days. Could you try reducing the number of shards to ~32 or ~128 (by e.g. concatenating)? That might unblock you.

thanks for the quick response, I'll give your suggestion a try later today.
Meanwhile, let me show you the config file I used:
train_urls:
  - "https://huggingface.co/datasets/allenai/c4/resolve/1ddc917116b730e1859edef32896ec5c16be51d0/en/c4-train.{00000..01023}-of-01024.json.gz"
cache_dir: "gs://sparse-llm/data_cache/tokenized/c4_llama/"
tokenizer: "meta-llama/Llama-2-7b-hf"

I think 10 hours later this is what I see on Ray dashboard; it seems like most tasks are not making progress; this is on 240 cpus (TPU v4-8), I can try TPU v4-32 later too, which will look more like your original setup.

I think there's just something wrong if the number of shards is too high
(pile is 32 shards or so). i'm trying to fix
On Tue, Dec 12, 2023 at 8:47 AM Tian Jin @.***> wrote:

thanks for the quick response, I'll give your suggestion a try later today.
Meanwhile, let me show you the config file I used:
train_urls:

"https://huggingface.co/datasets/allenai/c4/resolve/1ddc917116b730e1859edef32896ec5c16be51d0/en/c4-train.{00000..01023}-of-01024.json.gz"cache_dir: "gs://sparse-llm/data_cache/tokenized/c4_llama/"tokenizer: "meta-llama/Llama-2-7b-hf"

I think 10 hours later this is what I see on Ray dashboard; it seems like
most tasks are not making progress; this is on 240 cpus (TPU v4-8), I can
try TPU v4-32 later too, which will look more like your original setup.
Screenshot.2023-12-12.at.10.45.44.AM.png (view on web)
https://github.com/stanford-crfm/levanter/assets/6410074/2435ec16-9e53-4092-a45a-c43bbf748d4d
—
Reply to this email directly, view it on GitHub
https://github.com/stanford-crfm/levanter/issues/383#issuecomment-1852419220,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACLINLAPNNN3SI2M3N5G3YJCDBNAVCNFSM6AAAAABAPBQ6FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGQYTSMRSGA
.
You are receiving this because you commented.Message ID:
@.***>

It appears that at least part of the issue is that the c4 dataset is marked as a json file, but is actually a jsonl file.
Renaming them appears to help relieve some of the memory issue.
But anyhow, for what it's worth, two tricks required to get c4 working are:

Concat dataset partitions to reduce shards.
Rename json files in c4 to jsonl.

Oh interesting. That's good to know! I guess I knew that sometimes people write json for jsonl (HF does this a lot). We should try to sniff out json vs jsonl automatically

We should try to sniff out json vs jsonl automatically

no promises but I can try to submit a pr for that.

oh thanks!
On Thu, Dec 14, 2023 at 9:12 AM Tian Jin @.***> wrote:

We should try to sniff out json vs jsonl automatically
no promises but I can try to submit a pr for that.
—
Reply to this email directly, view it on GitHub
https://github.com/stanford-crfm/levanter/issues/383#issuecomment-1856248783,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACLIMSV5FQHEYMEQK4GSTYJMXPRAVCNFSM6AAAAABAPBQ6FOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJWGI2DQNZYGM
.
You are receiving this because you commented.Message ID:
@.***>

If you're up for it, could you try this branchactor_pqueue_stable? On OWT it's maintaining about 50% cpu utilization, which is fast enough that it will stay ahead of the actual training.
Working on improving it more

I believe this should be fixed in main now that #383 is merged

@dlwh I believe it's working after fixing the file format sniffing issue with C4:
https://github.com/stanford-crfm/levanter/pull/389


FBX model with chinese characters fails

I have a FBX file with chinese node names etc. The export to glTF results in an invalid gltf file like:
{"accessors":{"Basic Wall
'Wall ' should be followed by a chinese name. Is there an easy way to add support for chinese characters?
Thanks!

Hi @yomomano
I have tried to build the project with FBX SDK 2017.0 and it get built successfully. When I run the exe it throws exception on the line
// The next line will call the exporter
bRet = pExporter->Export(_scene);
Can you please help me to get the program run and export FBX to glTF?
Thanks


Packet will leave node if destination is a service without backends

I see this behavior in 1.11, but from the source code:
https://github.com/cilium/cilium/blob/58e4081c3c2a6e052d7c5e3443be33dd6ae5024b/bpf/bpf_lxc.c#L1253-L1275
It looks like the behavior is still the same where DROP_NO_SERVICE is returned for the cases either service is not found or backend is not found in here: https://github.com/cilium/cilium/blob/master/bpf/lib/lb.h#L1505
I am wondering if this is the correct behavior. Should we drop the packet if we know it belongs to a service without backend, as opposed to allowing it to exit the node.

@anfernee Thanks for the issue. Are you saying the following does not result in a packet drop even if the DROP_NO_SERVICE is returned by the lb4_local()?
if (IS_ERR(ret))
    return ret;

I don't exactly know which line is causing the problem. If it's not expected behavior, I can dig more into this one. Looks like it's not, right?

@anfernee Thanks for the issue. Are you saying the following does not result in a packet drop even if the DROP_NO_SERVICE is returned by the lb4_local()?
if (IS_ERR(ret))
    return ret;

As @michaelasp noticed in https://github.com/cilium/cilium/pull/21539, we don't even get to this point. lb4_lookup_service() returns NULL for a SVC with 0 backends, so we skip all subsequent service processing.

Reopening given the fix was reverted due to a BPF complexity regression.


How to do clickeable elements?

You add :> and the link after the text, like start=>start: Begin:>https://www.example.org

Hi, Instead of clickable links to some web pages, is it possible to call a function? For example,
start=>start: Begin:>myfoo(1)

https://github.com/adrai/flowchart.js/issues/148#issuecomment-386868790

One way I did it was by selecting the svg elements by their id in js and caliing the addEventListener function.
"cond=>condition: Yes or No?\n"
This creates an HTML element with id="cond" for the box and id="condt" for "Yes or No?".
You can select the object:
var box = document.querySelector("#cond");
var boxText = document.queerySelector("#condt");
box.addEventListener('click', fun() {});
boxText.addEventListener('click', fun() {});

Hope this helps.

If you just use the following syntax, you can call a javascript function:
start=>start: Start:>javascript:test('foo');
Is there a security concern with this option ?

In theory yes: XSS => https://github.com/adrai/flowchart.js/issues/176

Thank you for answer. I plan to generate the flowchart definition in a python backend api. So to be safe, I would have to generate the javascript to set onClick eventlistener on all my dom objects (I would prefer not to have to generate javascript code in the backend).
It would be great if there was a way to set the onClick function instead of setting the "href"(so to have the option to set an href or a function on onClick eventlistener as a link, that would really make this framework perfect for SPA (reactjs, angular, ...)


[Milestone] v4

I've recently made a lot of changes that I'd like to incorporate into a v4 release. There's no risk of breaking changes because nobody uses the @master tag! #23 It's awesome that good versioning practices are so prevalent!
Here's a summary of some of the changes that I'd like to incorporate into this repo:

[ ] Use with: instead of env: like all other actions #27
[ ] No longer using rsync to put files into a git repo. Instead, we use $GIT_WORK_TREE to move the git worktree to the files, not the files to the git worktree! 😲 #31
[ ] Use a non-Docker based method for building the action. #32
[ ] Add a test.yml GitHub workflow to make sure that the action works in PRs! #33
[ ] #34
[ ] #35
[ ] Use a strategy: clone|init switch to either a) add a commit to the existing repo or b) create a single commit and force push that. #36

Some people might want the semantics of a single deployed commit instead of a continuously stacking uncontrolled git history. More options are better.
For large image blobs (popular in some auto-generated wikis!) the git history gets bloated by so much non-diff-able .jpg 10MB files that it becomes slow over time. Force pushing a single commit solves this.

[x] Keeping strategy: clone as the default is perfect.
[ ] #37
[ ] Use gh auth setup-git to manage git credentials. #38

Dealing with tokens seems complicated. Delegating to an official tool that just needs a GITHUB_TOKEN var set is great!
gh is ALWAYS available on runners. Windows, macOS, and Ubuntu.

[ ] Add a note in the readme about #9 #8 #26

Preprocessing links to make them accurate in both GitHub source view and GitHub wiki is a good idea! It's a great fit for another action. And there's already an action that fits this: https://github.com/marketplace/actions/strip-markdown-extensions-from-links-action

[ ] Added an update-tags.yml action to auto-create v4 tags from v4.0.0 releases and keep them always updated. There's an official action for this! When you release a vX.Y.Z tag it changes vX to point to it. #39

I'm looking for feedback on any of these features and changes. I plan on doing one of:

Pulling each thing out into its own PR. Lots of things would be dependent on other PRs getting merged.
Creating a v4 branch and pruning and trimming it to @Andrew-Chen-Wang's specifications to get it merged as one PR.

I know this is like 2 months late. 😬 But hey, better late than never!

sidenote: consider renaming master to main to be "hip" and "cool" 😆 https://github.com/github/renaming

todo: @jcbhmr -- add the improvements from the readme https://github.com/Andrew-Chen-Wang/github-wiki-action/blob/jcbhmr/README.md to this list to do and track issues and open prs for that stuff

@jcbhmr Thanks so much for continuing this work. Since you've managed to make things backwards compatible, I've opened the former rule up such that now you can open PRs and merge them yourself or commit directly to the base branch.
I think we should refrain from moving to main for now since some users are using master as the target version. Perhaps a year down the line.

I think we should refrain from moving to main for now since some users are using master as the target version. Perhaps a year down the line.

No worries! ❤ I wouldn't worry too much about Andrew-Chen-Wang/github-wiki-action@master though; there doesn't appear to be any public usage of it (github search) (sourcegraph search). (I don't know about private) #23

@jcbhmr Thanks so much for continuing this work. Since you've managed to make things backwards compatible, I've opened the former rule up such that now you can open PRs and merge them yourself or commit directly to the base branch.

I want to caution that "backwards compatible" might not be the right word. I've tried to stick with the core idea of "push a folder to the .wiki.git repo", but I've done so by:

Changing from env: to with:
Adding more options
Changing the names of some of the options (WIKI_DIR => path:, REPO => repository:, etc)
Changing the core architecture of this project (Docker+Bash => Bash)
Using GIT_WORK_TREE and GIT_DIR instead of rsync

These changes, while good and (IMO) improvements, are nonetheless a new version and require changing a config. IDK if that's "backwards compatible" by the definition? 🤷♂
      - uses: Andrew-Chen-Wang/github-wiki-action@v3
        # ⚠ We use the env: key to provide our inputs! See #27.
        env:
          # Make sure this has a slash at the end! We use wiki/ by default.
          WIKI_DIR: my-octocat-wiki/
          # You MUST manually pass in the GitHub token.
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          # These are currently REQUIRED options
          GH_MAIL<EMAIL_ADDRESS>          GH_NAME: actions[bot]

to
      - uses: Andrew-Chen-Wang/github-wiki-action@v4
        with:
          path: my-octocat-wiki
          token: ${{ secrets.MY_TOKEN }}
        env:
          GIT_AUTHOR_NAME: Myself
          GIT_AUTHOR_EMAIL<EMAIL_ADDRESS>
hence, the v3 => v4 upgrade version bump. Which, again, is recommended practice! Bumping versions on improvements to the core idea of the project is good and that's what version tags are for, right? 😜

TLDR: I want explicit ✅ that going from (left) to (right) is 👍 for v4. Just to make sure we are on the same page.

I plan on merging all the PRs and then @-pinging you about making a v4.0.0 release. Sound good?
#40  #41  #42  #43  #44  #45  #47

Yes this sounds good. Major version update sounds good. Can't remember if the PR for the README includes a header saying "Breaking Changes in v4" which can help people migrate. I noticed a migration section iirc so that's great.
Plan sounds good to me. Thanks again.

current status: it works as of 070db4423e74902abf6480a811234f81c64239da

@Andrew-Chen-Wang All done!
You can see it work here: 🥳

Here's what I need from you...

create a GitHub Release
use the name & tag v4.0.0 -- don't worry! the update-tags action will do the v4 tag for you!
click the "Generate release notes" button to use PRs to generate release notes

You can also include this text in the release textbox copy+paste:
✈ If you're using v3, check out our [v3 ➡ v4 migration guide](https://github.com/Andrew-Chen-Wang/github-wiki-action/discussions/50)
``

then you _could_ also:
1. change the repo description to match the readme slogan #46 `📖 GitHub Action to sync a folder to the GitHub wiki`
2. change the repo tags to not include `actions-docker`
3. if you could, move the #50 discussion to the "Announcements" section of the discussions (I can't since I don't have perms to do that)

@Andrew-Chen-Wang All done! It works! I'm drafting Release v4.0.0 right now! I also need to actually write a complete guide: #50

Here's the draft release.

I will hit "Go" when I finish writing #50

Actually scratch the idea of #50 with a migration guide announcement post. Instead, since it's so short, why not put that in the release post? 🤔 That's what release posts are all for after all, right? 😜

@Andrew-Chen-Wang Created release! https://github.com/Andrew-Chen-Wang/github-wiki-action/releases/tag/v4.0.0
🎉🎉🎉🎉

awesome thanks again!


Feature Request] Heroku compatible deploymend - listen PORT, add WAHA_BASE_URL for media

I'm attempting to deploy the Waha Plus image to Heroku, but I keep encountering the following error:

I'd appreciate any insights or suggestions on how to resolve this issue.
Thanks in advance!

Hi!
Looks like you need to map PORT environment variable to WHATSAPP_API_PORT https://waha.devlike.pro/docs/how-to/config/#common so WAHA starts listening on the right port

Probably we could add it in the project as well, looks simple.

Thanks for your attention.
Heroku dynamically assigns the PORT environment variable at runtime, so each time the app restarts, a random value is assigned. This makes it impossible to manually set WHATSAPP_API_PORT with a fixed value (since it's not static), and I can't use something like WHATSAPP_API_PORT=$PORT.

I can't use something like WHATSAPP_API_PORT=$PORT.

Heroku doesn't allow it? that's a shame
We'll handle it in WAHA 2024.10.1 release, no worries! :+1:

I really appreciate that. Thank you team 💪

https://waha.devlike.pro/docs/how-to/config/#common
PORT and WAHA_BASE_URL is available so you can set it in heroku :rocket:


Signing up

I deployed the app on heroku and I just hit the following screen with no sign up option, how can I sign up?

From the readme:

The first user in single tenant mode can be created by a rake task rake create_user. We'll ask you for your credentials.


display of generatedhtml

Dear @michaelnmmeyer —
At the moment, a DiplEd html that is already generated does not yet get shown on the .io site.
See DHARMA_DiplEdSarvavajrodayaCodex.html at https://github.com/erc-dharma/tfd-sanskrit-philology/tree/master/docs/diplEd, not shown at https://erc-dharma.github.io/#tfd-collection. Can you correct this? Thanks.

Thanks. But now the DiplEd is shown in the place for CritEd files and not in the place for DiplEd files. See screenshots.

fixed in the meantime, I see!


Add note about allow/whitelist to readme.md

I found that the Chrome extension is now blocked, and this is the best way I could find to get it enabled again.  I figured it was better to open a PR than an issue, but please don't be shy about requesting changes, or just closing this if you think there's a better way to handle it.
BTW, I found it blocked when I realized I'd lost control of meetings and the extension name was grayed out in the Extensions menu.  The switch showed it as enabled, but Chrome was blocking it.  Not sure if that's because of an admin policy or Chrome update.

Interesting! I haven't yet seen a functioning copy of the extension get forcibly removed by a Chrome update.
Was Developer mode turned off in your browser after installing the extension, or was it kept on the whole time?
Was this actually in Google Chrome, or was it Edge? (I see edge://extensions/ in the readme update.)

Sigh. My bad. It was in Chrome, but I generated the instructions in Edge. Don't ask.
Developer mode was on the whole time, but it didn't seem to make a difference.
Sorry for the slow reply. Had a bunch of random OSS interactions in one week and lost track.

Thanks @ChrisRegado not just for merging, but for all the work you do on this.  You've made each and every work meeting just a little better for me, and I appreciate it.

My pleasure, glad you like it! :)


WIP: Build, wrap and upload stubs as artifacts for PRs

By uploading built and wrapped stubs as artifacts for PRs, we can point tools and libraries like esp-serial-flasher and esptool to these in order to do integration testing.

With the current setup, the user would have to specify a link to every chip output artifact. Also, the artifacts are zipped, unlike with the releases. I will explore what can be done to make this more convenient.

Closing this as another way to override stub sources with local files was chosen for esp-serial-flasher instead.


GDPAC-1950 Ajwain jobHandler was not comparing options.jobCodes to jo…

…b.code before invoking handler

Coverage increased (+0.06%) to 78.148% when pulling e8f87d92042c82ccf7824bc51a682c6da95cf17c on GDPAC-1986 into f0dd78640733c44c74c36a568838df752697f72d on master.


added Asked by Open AI no. 366

Thanks for making the changes, but from here it seems like the problem is completely different?

Thanks for making the changes, but from here it seems like the problem is completely different?

Can you please elaborate me please. you mentioned different repo instead of this

As in the PR is pretty much ready to be merged, but are you sure that the problem you shared is the Daily Coding Problem 366?

As in the PR is pretty much ready to be merged, but are you sure that the problem you shared is the Daily Coding Problem 366?

I know that it's going to be 1 day up from 365 day but the main thing is that it's from Open AI and the repo is purely python based solutions, So that's why i'm contributing to it.