WIP new JSON for queries

Refs:

#2049

TODO:

[x] Read queries JSON
[ ] Implement error display with "ok": false and an errors key
[ ] Read queries HTML
[ ] Read queries other formats (plugins)
[ ] Canned read queries (dispatched to from table)
[ ] Write queries (a canned query thing)
[ ] Implement different shapes, refactoring to share code with table
[ ] Implement a sensible subset of extras, also refactoring to share code with table
[ ] Get all tests passing

:books: Documentation preview :books:: https://datasette--2053.org.readthedocs.build/en/2053/

Table errors page currently does this:
{
  "ok": false,
  "error": "no such column: blah",
  "status": 400,
  "title": "Invalid SQL"
}

Uncommitted experimental code:
diff --git a/datasette/views/database.py b/datasette/views/database.py
index 455ebd1f..85775433 100644
--- a/datasette/views/database.py
+++ b/datasette/views/database.py
@@ -909,12 +909,13 @@ async def query_view(
     elif format_ in datasette.renderers.keys():
         # Dispatch request to the correct output format renderer
         # (CSV is not handled here due to streaming)
+        print(data)
         result = call_with_supported_arguments(
             datasette.renderers[format_][0],
             datasette=datasette,
-            columns=columns,
-            rows=rows,
-            sql=sql,
+            columns=data["rows"][0].keys(),
+            rows=data["rows"],
+            sql='',
             query_name=None,
             database=db.name,
             table=None,
@@ -923,7 +924,7 @@ async def query_view(
             # These will be deprecated in Datasette 1.0:
             args=request.args,
             data={
-                "rows": rows,
+                "rows": data["rows"],
             },  # TODO what should this be?
         )
         result = await await_me_maybe(result)
diff --git a/docs/index.rst b/docs/index.rst
index 5a9cc7ed..254ed3da 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -57,6 +57,7 @@ Contents
    settings
    introspection
    custom_templates
+   template_context
    plugins
    writing_plugins
    plugin_hooks

Now that I have the new View subclass from #2078 I want to use it to simplify this code.
Challenge: there are several things to consider here:

The /db page without ?sql= displays a list of tables in that database
With ?sql= it shows the query results for that query (or an error)
If it's a /db/name-of-canned-query it works a bit like the query page, but executes a canned query instead of the ?sql= query
POST /db/name-of-canned-query is support for writable canned queries

I'm going to entirely split canned queries off from ?sql= queries - they share a bunch of code right now which is just making everything much harder to follow.
I'll refactor their shared bits into functions that they both call.

Or maybe...

BaseQueryView(View) - knows how to render the results of a SQL query
QueryView(BaseQueryView) - renders from ?sql=
CannedQueryView(BaseQueryView) - renders for a named canned query

And then later perhaps:

RowQueryView(BaseQueryView) - renders the select * from t where pk = ?
TableQueryView(BaseQueryView) - replaces the super complex existing TableView

I should split out a canned_query.html template too, as something that extends the query.html template.

I should have an extra called extra_html_context which bundles together all of the weird extra stuff needed by the HTML template, and is then passed as the root context when the template is rendered (with the other stuff from extras patched into it).

I think the hardest part of getting this working is dealing with the different formats.
Idea: refactor .html as a format (since it's by far the most complex) and tweak the plugin hook a bit as part of that, then use what I learn from that to get the other formats working.

Another point of confusion is how /content sometimes serves the database index page (with a list of tables) and sometimes solves the results of a query.
I could resolve this by turning the information on the index page into extras, which can optionally be requested any time a query is run but default to being shown if there is no query.

Worth noting that the register_output_renderer() is actually pretty easy to extend, because it returns a dictionary which could have more keys (like the required set of extras) added to it:
@hookimpl
def register_output_renderer(datasette):
    return {
        "extension": "test",
        "render": render_demo,
        "can_render": can_render_demo,  # Optional
    }

I'm abandoning this branch in favour of a fresh attempt, described here:

https://github.com/simonw/datasette/issues/2109

I'll copy bits and pieces of this branch across as-needed.


Select field option list browser default vs dropdown

Why does the select field use the browser default as option list and not the custom option list from the Dropdown component?

https://carbondesignsystem.com/components/select/usage/#live-demo
https://carbondesignsystem.com/components/dropdown/usage/#live-demo

Select menus are notoriously tricky to style via CSS, as each Web Browser implements them differently. When we created the Select component, we avoided trying to override these styles and left them as defaults so that we did not break the out-of-the-box form element compatibility that comes baked in with the browser native select.
If you open the Select in Edge, Chrome, Firefox, and Safari, you'll see four different menus. We could not satisfactorily align the styles across the ever-changing implementations, so we've left them untouched. That way, it will render consistently for users expecting to see their browser of choice default select.
Dropdown was created to create a cohesively designed select element; it's just missing some of the "built-in" accessibility and form verification/functionality a select has by default. Hope that helps clears that up!


Best way to Sniff the Cookie

what is the best way to sniff the PHPSESSID cookie. idk how to find it. I tried wireshark but with no success.
I'm sorry if this annoys you.

It's impossible to sniff HTTPS traffic with Wireshark because that's the reason HTTPS was created.
Use Fiddler or Charles Proxy, they will generated a Root Certificate to allow HTTPS sniffing. Although there are some limitations, you should be able to get the cookie at least.

Your device is able to decrypt traffic and you can get the shared secret of the tls session with wireshark


[FeatureRequest] VsCode Command for checking if azurite blob service is running or start service only if service is not running

Which service(blob, file, queue, table) does this issue concern?
blob
Which version of the Azurite was used?
Azurite.azurite v3.30.0
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
Visual Studio Code Extension
What's the Node.js version?
v20.12.2
What problem was encountered?
I am from azure function team, we are building an vscode.dev extension which needs to start azurite blob service as part of our extension startup experience for customer every time reloading extension. we dont want to start blob service if it is already running nor see the error vscode window. right now dont see thats possible in vscode azurite commands.
Steps to reproduce the issue?

build an extension which installs and invoke vscode.commands.executeCommand('azurite.start_blob'); to be called every time page refresh once.
refresh the page you will see  the error message

If possible, please provide the debug log using the -d parameter, replacing <pathtodebuglog> with an appropriate path for your OS, or review the instructions for docker containers:
-d "<pathtodebuglog>"

Please be sure to remove any PII or sensitive information before sharing!
The debug log will log raw request headers and bodies, so that we can replay these against Azurite using REST and create tests to validate resolution.
Have you found a mitigation/solution?
No. We would need help if azurite vscode extension can :

add a new command to check if azurite service is already running
change existing start commands to allow an option to start if only if not running
a way to suppress the popup window for detecting address already in use.
suggest any workaround.

@YunchuWang
Thanks for the suggestion!
However this might not so appliable to Azurite.

We can start several Azurite instance on same host machine, with different ports.  So we should not block start Azurite instances when a machine already has Azurite started.
Azurite has several distribution package currently, like npm, docker, VScode extension. Even we add detection for Azurite started in VS code, VS code might don't have permission to detect if Azurite started with docker of npm.
The reason why this Azurite can't be start is caused by the port is occupied (it can be occupied by other process, not only existing Azurite instance). Azurite might don't have permission to detect if the port is occupied by another Azurite or other process. So we just report the real reason for the error.
We need to show the error when Azurite not start successfully, or other customer might be confused as Azurite not started and no error happen.

One workaround might be: detect if the Azurite port (by default: 10000,10001,100002) is occupied, if so, don't start Azurite. With this way, you need to make sure only Azurite can occupied the related ports on your client machine.


Uncaught Error: ENOTDIR: not a directory, scandir 'D:/01_Atom-Space\bpa-security\website-work\the...

[Enter steps to reproduce:]

...
...

Atom: 1.16.0 x64
Electron: 1.3.13
OS: Microsoft Windows 10 Pro
Thrown From: Remote-FTP package 0.10.7
Stack Trace
Uncaught Error: ENOTDIR: not a directory, scandir 'D:\01_Atom-Space\bpa-security\website-work\themes\HumanRights'
At fs.js:951

Error: ENOTDIR: not a directory, scandir 'D:\01_Atom-Space\bpa-security\website-work\themes\HumanRights'
    at Error (native)
    at Object.fs.readdirSync (fs.js:951:18)
    at Object.fs.readdirSync (ELECTRON_ASAR.js:571:28)
    at Client.traverseTree (/packages/Remote-FTP/lib/client.js:596:31)
    at /packages/Remote-FTP/lib/client.js:672:18
    at tryApply (/packages/Remote-FTP/lib/connectors/ftp.js:16:14)
    at error (/packages/Remote-FTP/lib/connectors/ftp.js:125:13)
    at /packages/Remote-FTP/lib/connectors/ftp.js:147:33
    at final (/packages/Remote-FTP/node_modules/ftp/lib/connection.js:476:11)
    at Socket.ondone (/packages/Remote-FTP/node_modules/ftp/lib/connection.js:451:7)
    at Socket.g (events.js:286:16)
    at emitNone (events.js:91:20)
    at Socket.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Commands
     -2:57.9.0 remote-ftp:connect (input.hidden-input)
     -2:48.6.0 remote-ftp:sync-with-remote (span.name.icon.icon-file-directory)
     -0:05.5.0 core:paste (input.hidden-input)
     -0:04.3.0 editor:newline (input.hidden-input)

Non-Core Packages
atom-beautify 0.29.22
Remote-FTP 0.10.7

Related issue #112


Add array_reverse function to datafusion-function-* crate

Which issue does this PR close?

Relate #9285
Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Windows CI test failure seems unrelated


Fix figures

Added functions and tried to fix collisions, didn't work. Tried to add random shapes. Didn't work..

All bugs to figures seems to be fixed, they can move left, right, down and rotate. Looks good!


Basename is not used in createHref

I use a basename in my router setup. In one of my components I have a Link to="/". The href attribute points to http://${hostname}/ instead of http://${hostname}/${contextPath}. The click handler works fine though.

Pass it in as a prop to BrowserRouter or HashRouter.

I'm doing that, but it seems to be ignored for the href.

I'll double-check when I'm at a computer, but it's got tests at least: github.com/ReactTraining/react-history/commit/b61ae2c362a34fe6545b24f5f06c77d23ab050b7

I do about this:
<Router basename='/foo'}>
  <Match exactly pattern="/" render={()=><Link to="/">Index</Link>} />
</Router>

The link renders as  but I expect it to render as as .

My comments in #3843: https://github.com/ReactTraining/react-router/pull/3843#issuecomment-247028856

@chentsulin

It seems that history is no longer responsible for creating hrefs. See mjackson/history#365 (comment)
And Link component just use router.createHref to create hrefs, so miss the basename: https://github.com/ReactTraining/react-router/blob/v4/modules/Link.js#L91

Ah, shazbot. Well, that's a little confusing as to who should be responsible then? Let me see what the thought process is on that and we can either re-open that PR or integrate the ideas in a more clear API. I feel like history should be aware of it, but we'll see if that stops at react-history or not.
Either way, thanks for this issue!

Just to clarify where responsibilities are: history has no knowledge of queries, so router is responsible for creating hrefs. It does know about basenames, but all locations you get in the history.listen callback already have properly-truncated pathnames.

Started a fix over in #3858. Need to do the latter part of what @mjackson said.

Now I'm confused. Consider the following:
<Router basename='/foo'>
  ...
  <Link to={"bar"} /> // renders `href='/foo/bar'` but transitions to "/foobar"
  <Link to={"/bar"} /> // renders `href='/bar'` but transitions to "/foo/bar"
  ...
</Router>

@jochenberger That is not how it should end up behaving. This area of the code should be considered "in progress".

Yes, I thought so, that's why I let you know. ;-)

I'm looking into this today.

I see similar issue on react-router@2. should I open separate issue?

Is there any progress on this?  I've noticed that the behavior for this in the 4.0.0-alpha.4 package is incorrect.  The docs reflect this as well.
basename

The base URL for all locations. If your app is served from a sub-directory on your server, you’ll want to set this to the sub-directory.

<HashRouter basename="/calendar" />

// now links like this:
<Link to="/today"/>
// will generate links with an href to "#/calendar/today"

That last comment is the issue.  If my application exists at http://example.com/calendar/, then what should be generated is a link to /calendar/#/today, not #/calendar/today.  As it stands now, the actual behavior is as documented.

@nibblesnbits If your app exists at http://example.com/calendar/, then a link within your app to #/today is a link to http://example.com/calendar/#/today, no basename is necessary.
Basenames are used to change the root url (location of /). Everything before the root doesn't matter to RR. For the <HashRouter>, #/ is the root url, so http://example.com/calendar/ doesn't matter.

Hey @pshrmn, basename is required for that /calendar thing because it is a subdirectory and not the actual root of the webserver.

07725d7ef566a9a4bcfcf91b1f0816dd1f546393 fixes this? let me know if it doesn't and we'll reopen.

 Is it already merged and published on npm? I found that this was merged a month ago but I experienced these issues a few days ago so in that case this commit wasnt helpful. Thanks in advance.

@patotoma, the easiest way I know of is to npm link from the repo.

Download the repo, and npm install its dependencies.
npm link the package so that you can use it from other projects on that machine.
npm install the version of react-router that you linked in you project and test as needed.


Creating docker machine hangs on 'Waiting machine to be running'

Creating docker machine hangs on 'Waiting machine to be running':
Running pre-create checks...
(test-test.new) Validating Vultr VPS parameters...
Creating machine...
(test-test.new) Creating Vultr VPS
(test-test.new) Waiting for IP address to become available...
(test-test.new) Created Vultr VPS ID: 17638002, Public IP: <IP_ADDRESS>, Private IP:
Waiting for machine to be running, this may take a few minutes...
Error creating machine: Error waiting for machine to be running: Maximum number of retries (60) exceeded
Container is created and its status becomes 'running' even before the docker-machine command fails. And in the vultr UI I can see it. Also I can ssh to the container.
Command:
docker-machine create --driver vultr --vultr-tag test-test.new --vultr-api-key=xxxxx --vultr-os-id=215 -vultr-region-id 6 --vultr-plan-id=201 test-test.new
OS: (tried both) Ubuntu 16.04 or Ubuntu 18.04
Driver: (tried both) https://github.com/janeczku/docker-machine-vultr/releases/download/v1.3.0/docker-machine-driver-vultr-Linux-x86_6
or
https://github.com/janeczku/docker-machine-vultr/releases/download/v1.4.0/docker-machine-driver-vultr-Linux-x86_6
Docker-machine: (tried both) v. 0.15.0, build b48dc28d or 0.14.0, build 89b8332

Same here,
docker-machine create --driver=vultr --vultr-tag odoo-vultr-1404.com --vultr-api-key 'U5H43FVNDZXPCRT7GTLMKBT7NL6BAECWRW7A' --vultr-os-id '160' --vultr-region-id '6' --vultr-plan-id '201' odoo-vultr-1404.com
Running pre-create checks... (odoo-vultr-1404.com) Validating Vultr VPS parameters... Creating machine... (odoo-vultr-1404.com) Creating Vultr VPS (odoo-vultr-1404.com) Waiting for IP address to become available... (odoo-vultr-1404.com) Created Vultr VPS ID: 17779292, Public IP: <IP_ADDRESS>, Private IP: Waiting for machine to be running, this may take a few minutes... Error creating machine: Error waiting for machine to be running: Maximum number of retries (60) exceeded The default lines below are for a sh/bash shell, you can specify the shell you're using, with the --shell flag.

+1
OS: Windows 10

I had this same issue but I got it working - eventually. I think its an SSH issue which means the connection drops midway hence the timeouts. These are the steps that resolved it for me - I'm not 100% if all are necessary...

Create Machine:

docker-machine create --driver vultr --vultr-api-key=<KEY> --vultr-region-id=8 --vultr-os-id=215 ubuntu

This will now time out with the error message above...

Stop the machine from the server or command docker-machine stop ubuntu

Start the machine with command docker-machine start ubuntu

SSH onto your machine: ssh root@<IP-get-it-from-Vultr> and run:

apt-get update
apt-get upgrade

Regenerate certificates:

 docker-machine regenerate-certs ubuntu

Regenerating TLS certificates
Waiting for SSH to be available...
Detecting the provisioner...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...

Now you should correctly have the docker daemon running.

Versions:
driver 1.4
docker-machine 0.16.1 (running from a mac)

Also having this issue on Ubuntu

@benscabbia your solution helped. thank you!


Add new features

Try to add new features in the project.

I have modified the background and also add the button to dynamically change the background color.
can you assign me this  under hacktoberfast.

I have completed the above task if you assign me then I will make a pull request.

sure


pub.dartlang.org package uploads aren't setting the latest version

When a package is uploaded to pub.dartlang.org, it should set the "latest version" property of the package object to the newly-uploaded version. This doesn't seem to be happening, which causes the description, authors, and homepage of the package to be inaccessible.

This issue has been moved to dart-lang/pub#158.


test: add unit test for flag trigger

This pull request primarily includes adding unit tests for FlagTrigger and changes to the Bucketeer project's "pkg/feature/api" package. This change focuses on refactoring the FeatureService structure and related methods in api.go and flag_trigger.go to use the new v2fs package for feature storage. Additionally, error handling has been simplified for several methods, and the "generateTriggerURL" method no longer returns an error.
Here are the most important changes:
Add unit test for flag trigger.
Refactoring to use v2fs package:

pkg/feature/api/api.go: Imported the v2fs package and its FeatureStorage and FlagTriggerStorage types. These were then added as fields to the FeatureService struct, replacing direct usage of the mysqlClient. The NewFeatureService function was also updated to initialize these new fields. [1] [2] [3]
pkg/feature/api/api_test.go: Similar changes were made in the test file, where mock instances of FeatureStorage and FlagTriggerStorage are now created and used in the createFeatureService and createFeatureServiceNew functions. [1] [2] [3]

Simplification of error handling:

pkg/feature/api/flag_trigger.go: In several methods, the error handling for generateTriggerURL has been removed, as this function no longer returns errors. Instead, the URL is directly assigned to triggerURL. Additionally, error handling in ResetFlagTrigger and DeleteFlagTrigger has been simplified to return a single error status. [1] [2] [3] [4] [5] [6] [7] [8] [9]

Refactoring of FeatureService methods:

pkg/feature/api/flag_trigger.go: All methods in this file that previously created new instances of FlagTriggerStorage or FeatureStorage now use the instances stored in the FeatureService struct. This includes the GetFlagTrigger, FlagTriggerWebhook, and generateTriggerURL methods. [1] [2] [3]

@kentakozuka
Thank you for your suggestions!
I adopted it because it was necessary.
https://github.com/bucketeer-io/bucketeer/pull/773/commits/dbad210cda8cbe0adde84f32688a1ecdddc005be
https://github.com/bucketeer-io/bucketeer/pull/773/commits/f80df540e9a3da3d034d78b262848f9ac9f43661

Thank you for the reviews!


FileNotFoundError: [WinError 2] The system cannot find the specified file

I get this error message every time I want to download a course and now I don't know how to proceed:

used --skip-hls


Wrong relationship assignment

Hi,
I just ran into an issue where two relationships between two types are not assigned properly. In my example I have a "House" and a "Human" with relationships:

House
{
    "id":0,
    "residents":[{
        "id":0
    }]
}

I expect this to insert a House and a Human where the human is a resident of the house. Owners of the house should be left nil.
In my case House and Human was added correctly with the relationship between residents and residenthouse but it also created a relationship between owners and ownhouses with the same instances after syncing.
Is there something I need to setup to avoid this behavior?

Oh god, I'm sorry that I didn't had the chance to look at this earlier.

I'll double check.

I was able to reproduce this bug.

Fixed! To explain it a little bit better.
Sync first will check the houses in the JSON. It will find one house with id 0. It will create the house, then it will move to residents.
Sync then, will go through all the residents, it will find one with id 0. It will make a Human entry and will connect the residenthouse with the residents. Then it will continue the loop and check for ownhouses and this is where the bug was located.
Sync wasn't very good at handling Entities that point to each other multiple times, so it would think that both ownhouses and residenthouse in Human are the same.
I fixed this issue by sending which relationship was supposed to be evaluated for an array of residents. Meaning if I'm parsing residents, then I should only connect residenthouse and ignore ownhouses.

Published fix here https://github.com/hyperoslo/Sync/releases/tag/1.12.2


XYPlot defines x and y domains in such a way that line does not show on plot

When I pass simple xy data in string form to any of line series, the line series is wrong because domains are being calculated in a way that does not seem to have any predictable behaviour.
I have example list of 2000 xy pairs. Plot would render differently depending of how much of data I take. Example pens:

charts renders only 100 points when I have 200
there is no line at all if have 2000 points

It works fine for first 100 pairs, then show only 100 for 200, but show 1000 points fine again.
It will set xDomain to 1000 for anything grater than 1000, but y domain would change to strange values, with the strangest being in second example - 2000 pairs, no line visible and lower value being higher on y axis.

Hi, thanks for providing codepen links.
Here the weird behavior is due to the fact that your x values are strings. The maximum value of your x in the first example is therefore "99" (as "99" > "100", etc.)
Strings are a valid option for x attributes (ie for ordinal scales, etc.)
try converting your data into numbers, you can also do that directly in the data property (ie <LineSeries data={data.map(d => ({x: Number(d.x), y: Number(d.y)}))} />
Hope it helps?

Could also do an accessor to fix the values on the fly (something like ‘getX={obj => Number(obj.x)}’) however it is generally preferable to do make the change all at once

I would not expect it to work correctly if the chart would use lexicographic order, but it is definitely not doing that. Both, scales and values are using numeric order, not the lexicographic. If I pick just 10 values, it would even render 0.5, 1.5 etc - values that do not exist in the data. If I remove some values it would fill up the holes.
So for me it is just surprising behaviour - if the plot is numeric in handling of values, why would domain be established using lexicographical order?
Changing the data to a number would obviously work for this example, both needs to be changed to get proper results.  It is not always an option though as even "long" values can't be send as numbers in JavaScript because of too low max safe integer (2^53-1). This can be a problem, for instance, for nanosecond precise timestamp. I believe d3 can support those.

Thanks for the fast response 👍


Create exampleSite folder

Discussed in https://github.com/colinwilson/lotusdocs/discussions/3

Originally posted by hugo-sid January 14, 2023
Hi @colinwilson . From the screenshot, lotus docs looks exiting.
Any plans to have the Hugo exampleSite folder ? So, that one can easily check out how to organise content.

Create an exampleSite to demo Lotus Docs' feature set

Resolved


Broken link in the landing page

There is a broken link named as PyPath adapter at BioCypher Landing page under the host Module Adapter title. I think it should be corrected as this.

good catch, didn't realise denes had already merged the major refactor of directory structure to pypath.


Update how task results are decoded & how the upload request for download is done

This PR fixes two issues, increases the supported certificate version, adds more verbose messaging around errors, and also logs any external IP changes:

If cat is run on a non-utf8 encoded file, the server won't be able to parse the task results. The approach in this PR uses fallback encodings to see if we can decode the result with any other encodings, and if we can, re-encodes everything with utf8.

For my use cases, shift_jis and euc-jp are enough. The exact order and number of fallback encodings are up for debate and this could be a config option, so users who don't need the flexibility can disable it.

The download command gzips and encrypts the target file before uploading it to the server. Since encryption happens after compression, the request content is base64 instead of gzipped bytes. If we have the Content-Encoding: gzip header when the content is base64, services such as AWS Lambda where a request is fully parsed before getting passed to the backend, the request will fail with 415 Unsupported Media Type. Removing the Content-Encoding: gzip header fixes this issue.

An alternative would be reversing the compress/encrypt order, but compressing an encrypted blob would yield bigger file sizes.

The current server allows https connections from SSLv3 upwards. Rust clients can complain about this, and since we control both server and client, there's no need to support such old methods. This PR updates ssl_version to TLS 1.2.

Currently, it's difficult to debug why a request was rejected or where an exception occurred. I have increased the verbosity of all_exception_handler with a dump_debug_info_for_exception method and updated how notifyBadRequest works to give better guidance on why a request might have been rejected.

Logging changes in external ip address would allow better accounting, since as it is now, we'd only have access to the latest external ip.

This is great! Thank you so much for getting rid of some of the technical debt 😅. Great fixes, and wonderful implementation! 💯


Wrong return type in phpdoc for getSessionExpiration

In OneLogin_Saml2_Auth on line 397 stats that the return value will be DateTime|null however checking where the property _sessionExpiration is set it actually gets set as an int.

I'll make a quick PR changing it from DateTime to int to be consistent with what's being returned currently.

PR #288

Fixed in #308


Fix lack of graceful shutdown for notification task

cc @C-Loftus @albertotirla
Can you confirm that this fixes the "Odilia doesn't quit with Ctrl+c" bug?

Yes it works. Thanks!!


Unique index not always generating

unique: true property seems not to be functioning. I have tried using it with index: true combination, but no luck 😕
Here's what I got:
import { Field } from 'payload/types';
import { merge } from 'lodash';
import FormatSlug from '../hooks/formatSlug';

type SlugType = (
  options?: {
    fieldToUse?: string,
    overrides?: Record<string, unknown>,
  }
) => Field;

const Slug: SlugType = ({
  fieldToUse = 'title',
  overrides = {},
} = {}) => {
  const dynamicSlug: Field = {
    name: 'slug',
    label: 'Slug',
    type: 'text',
    unique: true,
    admin: {
      position: 'sidebar',
    },
    hooks: {
      beforeValidate: [
        FormatSlug(fieldToUse),
      ],
    },
  };

  return merge(dynamicSlug, overrides);
};

export default Slug;

Properly added to fields list within my collection, e.g.
fields: [
  ...
  {
    name: 'title',
    type: 'text',
    required: true,
    index: true,
  },
  Slug(),
  ...
]

From admin/cms perspective everything is fine. The main problem is that for some reason I am missing index with unique property for my slug field, and to top it up, can't see index for title field either. Have tried numerous things, even dropping collection table, with hopes that freshly generated one will be fine, but unfortunately no cigar.

Payload v1.6.3

Interestingly, I have just randomly added additional field to my collection
    {
      name: 'random',
      type: 'text',
    },

and that made it generate all missing indexes

I believe steps to recreate are:

create collection
let payload generate table for it
add slug field

or

create collection
let payload generate table for it
drop table for the collection
add slug field

I can't reproduce your issue.
Your steps dont match this, but the only thing thing I can think is that Payload won't replace an existing index, even if it changes. For example if your collection already had index: true on your slug field, then you started again after changing it to unique: true. In that case you'd still see the original (non-unique) index until you delete the index (or entire collection) and started again.
Having both index and unique true is the same as having just unique: true.
What version of Mongo are you using? Have you set custom settings for your mongoose connection? I can't think of anything else that would be a factor here.
It is not ideal, but you can always manually manage your indexes as a fallback to having the generated ones.

Not on my laptop at the moment, but I'll have another go this evening, will see if I can provide more accurate steps 👍

I'm going to close this for now since I can't do anything more for it. Feel free to re-open and let me know how it goes!


[BUG] API "Nancy self host was unable to start" error in logs

Please confirm these before moving forward

[X] I have searched for my issue and have not found a work-in-progress/duplicate/resolved issue.
[X] I have tested that this issue has not been fixed in the latest (beta or stable) release.
[X] I have checked the FAQ section for solutions.
[X] This issue is about a bug (if it is not, please use the correct template).

UniGetUI Version
3.1.0 beta 1
Windows version, edition and architecture
Windows 11 Pro
Describe your issue
Not sure what this means, but maybe it should be brought to your attention. See the logs.
Steps to reproduce the issue
-
UniGetUI Log
...
[01/07/2024 17:22:29] Randomly-generated background API auth token for the current session: 206m3w9k0dnbfubg38gwfovrf4q28uply2ti6v9p0013tfhd61v6a645wdx3cpdm
[01/07/2024 17:22:29] An error occurred while initializing the API
[01/07/2024 17:22:29] Nancy.Hosting.Self.AutomaticUrlReservationCreationFailureException: The Nancy self host was unable to start, as no namespace reservation existed for the provided url(s).

                      Please either enable UrlReservations.CreateAutomatically on the HostConfiguration provided to
                      the NancyHost, or create the reservations manually with the (elevated) command(s):

                      netsh http add urlacl url="http://+:7058/" user="Everyone"

                         at Nancy.Hosting.Self.NancyHost.StartListener()
                         at Nancy.Hosting.Self.NancyHost.Start()
                         at UniGetUI.Interface.BackgroundApiRunner.Start() in C:\SomePrograms\WingetUI-Store\src\UniGetUI.Interface.BackgroundApi\BackgroundApi.cs:line 73
...

Package Managers Logs
\-

Relevant information
No response
Screenshots and videos
No response

Are you running 2 or more parallel instances of UniGetUI?

Do you know if you use any program that uses the port 7058?
Can you check if your firewall is blocking localhost:7058

Nancy is used for "Widgets for UniGetUI" and  the "Share Packages" feature, UniGetUI should be able to work without it (in fact, it can be disabled from the settings).
If it is only localhost there should be no need to reserve the port, but perhaps EaseUS software already has this port reserved for itself


4.0.0 release missing

The last release is 3.0.0 from July 2018
https://github.com/mattjohnsonpint/SimpleImpersonation/releases

Fixed.  Thanks.


New instances don't have an eth0 interface

The instances created by this module don't have an eth0 interface so the forwarding rule isn't working.

I signed it!

My instances appear to have eth0 interface:
admin@nat-gateway-us-central1-a-dr6f:/$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 42:01:0a:00:5c:07 brd ff:ff:ff:ff:ff:ff

My instances appear to have eth0 interface:
admin@nat-gateway-us-central1-a-dr6f:/$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 42:01:0a:00:5c:07 brd ff:ff:ff:ff:ff:ff

what image did you use? we used the ubuntu image.

Maybe the right thing to do, if it's different per image, is to make it configurable.

Yeh I was using the default debian image: https://github.com/GoogleCloudPlatform/terraform-google-nat-gateway/blob/master/variables.tf#L79


Support for external python management tools

Hello,
It would be really handy if uv was able to understand and use other tools (for example asdf/pyenv) for the discovery and installation of Python rather than only python-build-standalone.

Related:

https://github.com/astral-sh/uv/issues/6479


Support metrics exporter with oc exporter config

Also remove outdated v1 configs and fix tests.
Updates #231.

Codecov Report

Merging #232 into master will decrease coverage by 0.49%.
The diff coverage is 44.89%.

@@           Coverage Diff            @@
##           master    #232     +/-   ##
========================================
- Coverage   72.79%   72.3%   -0.5%
========================================
  Files         110     110
  Lines        6338    6300     -38
========================================
- Hits         4614    4555     -59
- Misses       1491    1513     +22
+ Partials      233     232      -1

Impacted Files
Coverage Δ

exporter/opencensusexporter/opencensus.go
30.18% <0%> (-38.33%)
:arrow_down:

exporter/opencensusexporter/factory.go
80% <64.7%> (-5.9%)
:arrow_down:

service/builder/exporters_builder.go
75.71% <0%> (-14.29%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b39842b...59a2c20. Read the comment docs.

Good point, added documentation in https://github.com/open-telemetry/opentelemetry-service/pull/232/commits/71a1583c02e779050f2579e3de34154172d736d9.


Missing Palauan word: deraiuechasu

Missing Palauan word: deraiuechasu created by<EMAIL_ADDRESS>on 2021-02-02 09:19:23

<EMAIL_ADDRESS>replied, Please see below and confirm/reject. Also, which spelling?

<EMAIL_ADDRESS>replied, Teraiuechasu.
It definitely starts with a "t"

<EMAIL_ADDRESS>replied, Thanks.  Added.  Closing.  "a kind of blindsnake that looks like a worm. Lives in old leaves."

<EMAIL_ADDRESS>replied, Wait. Is that the definition? A kind of a blind snake that looks like worm,
lives in leaves?
I didn't see that in the original email.
Teraiuechasu is bioluminescence.
So maybe deraiuechasu is this other organism...
Sorry!

<EMAIL_ADDRESS>replied, Whoops!  I’ll reopen the Issue.  Looks like we need confirmation of both of
these words now!  :)
Teraiuechasu means bioluminescence?  Can anyone confirm please?
Teraiuechasu/deraiuechasu means blind snake that looks like worm and lives
in leaves?  Can anyone confirm please?
Sorry, Lukes, that was my mistake of not forwarding the entire original
email!

<EMAIL_ADDRESS>replied, Teraiuechasu I know for sure. Some nights, a trip out for line fishing is
cut short due to bioluminescents in the water. They would say "Ng kmal klou
a teraiuechasu meng mekngit el dongereel."
The only time I hear this word used is when they refer to bioluminescents
in the water, specifically when you see the sparkles or glows IN the water
due to some movement. I've never heard one use the word to refer to the
bioluminescence in, say, a jelly fish or firefly or fungi.
Extra note: There's a secluded spot in Koror, near Nikko Bay, that we've
gone to a couple of times to swim at dusk to early evening because of the
bioluminescents. When one dives in the water, treads in the water, swim
around, they're surrounded by them. It's quite a spectacular sight
actually.
Anyway, hope this helps. I'll ask about deraiuechasu.


Handle the possibility of two table creation requests conflicting each other

Handle the possibility of two table creation requests conflicting each other.  The two tables could be created with the same version number.
Design considerations:
An initial thought was to use a SERIAL type on the version number, but the version resets for each table.
Acceptance criteria:

The lookup table creation code can handle two or more simultaneous requests to create a table.

The constraints on the database will not allow the same table name with the same version number to be created, so this scenario is not possible thanks to that constraint.


OLS-598: Script to delete OLS archive from Ceph bucket

Description
Script to delete OLS archive from Ceph bucket
This is the last thing that is missing to close OLS-598
Type of change

[ ] Refactor
[x] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[x] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #OLS-598
Closes #

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.15%. Comparing base (59dab1d) to head (4b52751).
Report is 20 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #903   +/-   ##
=======================================
  Coverage   95.15%   95.15%
=======================================
  Files          55       55
  Lines        2045     2045
=======================================
  Hits         1946     1946
  Misses         99       99

/lgtm

/approve


Fix foreign key constraints

Resolves #784

Description
Looking into #784, constraints strip off config that dbt hasn't specified, so parent and parent_columns were not making it to the macro.  It looks as though 'to' and 'to_columns' can fill this role instead.  Will need to update any documentation out there that references parent and parent_columns.
Checklist

[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR
[x] I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.

Need a little more context to understand, seems we are only changing what we expect the users to provide for the foreign key table and column. You mention parent and parent_columns are not working because of dbt will strip it out? Does to and to_columns work because that's a supported keyword by dbt? Any documents reference?

Need a little more context to understand, seems we are only changing what we expect the users to provide for the foreign key table and column. You mention parent and parent_columns are not working because of dbt will strip it out? Does to and to_columns work because that's a supported keyword by dbt? Any documents reference?

Exactly.  And no, these columns are not referenced in the docs I can find, but are public fields on the ModelConstraint class.


how to select coordinate points in the Splat scene

There is an issue that has been bothering me, which is how to select coordinate points in the Splat scene through mouse clicks in order to conveniently add regular meshes to the scene. However, when I tried to implement it using the raycaster.intersectObjects() method, it didn’t work.

Since the splat geometry is calculated in the vertex shader, a custom ray caster will be needed for this functionality. I do plan on writing one when I have time :)

I will keep an eye on it and hope to complete it

I was trying the same! Will keep trying and share it if I reach the solution :) How where you thinking on developing it @mkkellogg ??

I added a quick-and-dirty raycasting implementation. You can press C to display a mesh cursor that shows where your mouse cursor is projected onto the mesh. You can press I to toggle an info panel that shows the 3D coordinates of that mesh cursor.


Adding list docs.

Also making a number of changes to the snippets, to bring them in
line with the current implementation of the list component.
@Garbee PTAL!

LGTM!


What are "commit_weight" and "fee_per_kw"

Hi Everyone,
A newbie question, in layman terms what are "commit_weight" and "fee_per_kw" for?
Best,

Commit weight is how large the commitment transaction is in weight units. And the fee that was chosen when crafting the commitment transaction is expressed in per_kw = sats/kWU = sats/1000 weight units.


509 pjg clim departures

@doutriaux1 @durack1   All intended changes are to mean_climate_metrics_calculations.py
... not sure what jupyter stuff is coming from... it can be removed before merging if that make sense

@gleckler1 do we want this one?

@doutriaux1 funny you should ask, I was just looking at this yesterday.  The answer for the two file changes is yes.  I don't know what the notebook is - I will take a look at it.

@gleckler1 I merged master in this one as well and fixed conflicts.

@gleckler1 in effort of housekeeping, could you please check what's the status of this branch? Is this PR still valid?

@gleckler1 could you remind me if this PR is still valid?

@gleckler1 I guess this PR is inactive and should be okay to close it without merging, is that correct?

@gleckler1 I changed this PR to draft status. It looks like key addition in this PR is to add "rms_devzm_xy","rms_devam_xy" statistics for the mean climate metrics. I will consider adding them to the xcdat version mean climate later.

I am closing this PR and will reopen a new one once it is ready for #897.


Clarified instructions (Copy of #80)

Instructions weren't inherently wrong, but if you close the Advanced Google services tab before clicking 'Okay' (i.e. when you open Google API) it doesn't save.
Copy of #80 https://github.com/GioBonvi/GoogleContactsEventsNotifier/pull/80#issuecomment-334549457 but for the development branch.

Yes, that's it.
Thank you again for your contribution!


Implicit conversion on unmarshalling

Hey,
I'm using spray-json with akka-http to parse and unparse json objects.
The problem im trying to unmarshall is Array[Array[String]]. I have an implicit conversion function that convert this kind of object to Array[TimeRange](Time Range is a class I made).
I want the spray library to automatically use my conversion function whenever it trys to unmarshall the original object.
I hope you understand.
Thanks for the help

That's something you could build on top of spray-json, but I don't think it belongs into the core.
Just provide implicits or methods which would do that.
We do so in Akka HTTP - Unmarshal(bla).to[MyValue] etc.

Thanks for the response, but this does not quite help me.
I'll try to explain it more clearly.
I want to convert the class type im getting from the request to my type which is a little different.
Im receiving class A with few properties. One of the properties is of type Array[Array[String]](lets call it X). I'm trying to parse the json to class B which is like A but instead of property X it has property Y which is of type Array[TimeRange]. How do I manage to make akka automatically unmarshal the object as Type A and then converts it to type B.
Thanks


Upgrades202409 part2

Ref https://app.asana.com/0/1204880927741389/1208303037023014/f
Closes:
https://democracy-club-gp.sentry.io/share/issue/4a5e72f0cdf341ae943d0596289f97bf/
https://democracy-club-gp.sentry.io/share/issue/41a20ab06e774a9b8aeccce5f39a81d2/
https://democracy-club-gp.sentry.io/issues/5899045389/?environment=staging&project=163593&query=is%3Aunresolved+issue.priority%3A[high%2C+medium]&referrer=issue-stream&statsPeriod=90d&stream_index=0
co-authoured with @symroe
BEFORE MERGING:

[ ] Remove staging branch

This PR implements upgrades to Python 3.12/Django 4.2.15/Postgres 16 as well as implements fixes to the staging deploy that were not caught in the previous PR.
The changes @symroe made to StaticStorage in https://github.com/DemocracyClub/electionleaflets/pull/255/commits/4cbc24c482ede6fbd22ba833d78baee640abb8c3 are from DC utils. Once these changes are merged, we should add dc utils and rip that code out of this project.
In addition to these changes, the DNS for [stage.electionleaflets.org (http://stage.electionleaflets.org/) wasn't hooked up. The prod EL account didn't have a stage NS set up. Sym added a NS record to electionleaflets.org pointing at the hosted zone in the stage EL account (that is set up).
Up next:

[ ] Review this list https://github.com/DemocracyClub/electionleaflets/security/dependabot
[ ] Install dc_utils to handle static files storage settings

I've left some comments to follow up on here.
I think there is a further issue though, which is that I think we're probably duplicating requirements in the main Pipfile and deploy-requirements.txt and just installing everything everywhere rather than what we need where we need it.
I reckon to give proper actionable feedback on this I need to do a bit of a spike so I'm going to do that first but you might as well get on with the easy comments while I am doing that.

Having a look over staging there's a couple of things I noticed.

Dark text on dark background in the footer

This isn't the same on production. I assume we've changed something in this PR that breaks this?

If I look at for example https://stage.electionleaflets.org/leaflets/27/ it is linked to a party, but the party doesn't exist. In fact there's no parties at all https://stage.electionleaflets.org/parties/ I don't mind if this is just because the data on staging is a bit broken, but I'd like to have confidence that this isn't a problem we've introduced in this PR.

This might be the result of removing the ref from the design system package, I'll look into it.

OK. Here's my suggested diff for the packages:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index e58e7ac..1ba37d7 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -25,7 +25,6 @@ jobs:
           name: Set up virtual environment
           command: |
             pipenv sync --dev
-      - run: pip install -r deploy-requirements.txt
       - run: make lambda-layers/DependenciesLayer/requirements.txt
       - run: pipenv run collectstatic
       - run: npm install
diff --git a/deploy-requirements.txt b/deploy-requirements.txt
index 72869f1..37b7452 100644
--- a/deploy-requirements.txt
+++ b/deploy-requirements.txt
@@ -1,6 +1,2 @@
 aws-sam-cli==1.115.0
 urllib3==2.2.3
-Django==4.2.15
-git+https://github.com/DemocracyClub/design-system.git
-git+https://github.com/DemocracyClub/awsgi.git
-sentry_sdk==2.14.0

Basically:

We don't need to install deploy-requirements.txt in the install_and_update_dependencies because we're not trying to deploy anything yet. We only need that once we get to the build/deploy jobs
None of those other packages are needed in the deploy step

I guess with the extra packages in deploy-requirements.txt were hitting errors in the deploy step so you started adding more packages to deploy-requirements.txt, then added pipenv sync --dev which fixed them and didn't take the extra packages back out of deploy-requirements.txt ?
Ta

OK. Had a quick look at it. Here's my suggested diff for the packages:
diff --git a/.circleci/config.yml b/.circleci/config.yml
index e58e7ac..1ba37d7 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -25,7 +25,6 @@ jobs:
           name: Set up virtual environment
           command: |
             pipenv sync --dev
-      - run: pip install -r deploy-requirements.txt
       - run: make lambda-layers/DependenciesLayer/requirements.txt
       - run: pipenv run collectstatic
       - run: npm install
diff --git a/deploy-requirements.txt b/deploy-requirements.txt
index 72869f1..37b7452 100644
--- a/deploy-requirements.txt
+++ b/deploy-requirements.txt
@@ -1,6 +1,2 @@
 aws-sam-cli==1.115.0
 urllib3==2.2.3
-Django==4.2.15
-git+https://github.com/DemocracyClub/design-system.git
-git+https://github.com/DemocracyClub/awsgi.git
-sentry_sdk==2.14.0

Basically:

We don't need to install deploy-requirements.txt in the install_and_update_dependencies because we're not trying to deploy anything yet. We only need that once we get to the build/deploy jobs
None of those other packages are needed in the deploy step

I guess with the extra packages in deploy-requirements.txt were hitting errors in the deploy step so you started adding more packages to deploy-requirements.txt, then added pipenv sync --dev which fixed them and didn't take the extra packages back out of deploy-requirements.txt ?
Ta

Exactly. That's now fixed with https://github.com/DemocracyClub/electionleaflets/pull/255/commits/fb011d466d9f17983a1ab6f71c719e93e231c6d7.

UPDATE: The version of design-system used on master is the very first version. Since then, changes have been made to the design system that have not accounted for the child  elements which (I think) only appear in EL. This will require adding that flexibility to the design-system. Shall I put a temp fix in here?

Yeah I think lets just work round it here if we can. Can we bing the footer more into line with what we're doing on other applications or bodge it by putting it in a
 (even though that's not really the right class name) or something?
I reckon just binning that text from the footer is probably better than leaving the dark text on dark background.
Did you have a chance to think about the missing parties on staging at all?

UPDATE: The version of design-system used on master is the very first version. Since then, changes have been made to the design system that have not accounted for the child  elements which (I think) only appear in EL. This will require adding that flexibility to the design-system. Shall I put a temp fix in here?

Yeah I think lets just work round it here if we can. Can we bing the footer more into line with what we're doing on other applications or bodge it by putting it in a
 (even though that's not really the right class name) or something?
I reckon just binning that text from the footer is probably better than leaving the dark text on dark background.
I think this is the best solution to remove that text and add a link to DC website similar to other sites

Did you have a chance to think about the missing parties on staging at all?
Will look into this now.


URLs are not correctly encoded when publishing diagnostics

The spec says that URIs should be encoded according to RFC3986; incoming URIs are decoded in uriToFilePath but not encoded in filePathToUri.
(I am working on fixing this!)
See also: vscode-hie-server #47

Derp, github closed this issue after I merged the branch on my own fork.


CBS.com Late Show with Steven Colbert won't download anymore

Hello! I couldn't find an open issue for CBS or "Late Show", so I am asking you to look into this. As always I am really grateful this tool does exist and work so well!
Looks like CBS changed something on their website, I can't download the Late Show anymore.

C:\Users\hstracker90> youtube-dl http://www.cbs.com/shows/the-late-show-with-stephen-colbert/video/U6B47sA7bcL1OliOF9n_i6_llXVZLM6r/the-late-show-3-17-2016-william-h-macy-melissa-rauch-isaac-mizrahi-/ --verbose
[debug] System config: []
[debug] User config: [u'-o', u'C:/Users/hstracker90/Downloads/%(title)s.%(ext)s']
[debug] Command-line args: [u'http://www.cbs.com/shows/the-late-show-with-stephen-colbert/video/U6B47sA7bcL1OliOF9n_i6_llXVZLM6r/the-late-show-3-17-2016-william-h-macy-melissa-rauch-isaac-mizrahi-/', u'--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2016.03.14
[debug] Python version 2.7.10 - Windows-8-6.2.9200
[debug] exe versions: ffmpeg 2.8.git, ffprobe 2.8.git, rtmpdump 2.4
[debug] Proxy map: {}
[CBS] the-late-show-3-17-2016-william-h-macy-melissa-rauch-isaac-mizrahi-: Downloading webpage
[ThePlatform] of9zYH1kUpJS: Downloading SMIL data
Traceback (most recent call last):
File "main.py", line 19, in
File "youtube_dl_init_.pyo", line 412, in main
File "youtube_dl_init_.pyo", line 402, in _real_main
File "youtube_dl\YoutubeDL.pyo", line 1719, in download
File "youtube_dl\YoutubeDL.pyo", line 679, in extract_info
File "youtube_dl\YoutubeDL.pyo", line 736, in process_ie_result
File "youtube_dl\YoutubeDL.pyo", line 668, in extract_info
File "youtube_dl\extractor\common.pyo", line 320, in extract
File "youtube_dl\extractor\theplatform.pyo", line 234, in _real_extract
File "youtube_dl\extractor\theplatform.pyo", line 34, in _extract_theplatform_smil
File "youtube_dl\extractor\common.pyo", line 501, in _download_xml
File "youtube_dl\compat.pyo", line 248, in compat_etree_fromstring
File "youtube_dl\compat.pyo", line 237, in _XML
File "xml\etree\ElementTree.pyo", line 1642, in feed
File "xml\etree\ElementTree.pyo", line 1506, in _raiseerror
xml.etree.ElementTree.ParseError: syntax error: line 1, column 0

It's already fixed (#8427) and it will work in the next version.

sorry, the problem in this issue has been fixed but another error happen(some videos are served using brightcove once but the http formats doesn't work).
i just pushed another change to check brightcove once http formats.

your latest commit seems to be working although it included one of the ads (Mercedes Benz) with the file

the manifest contain unencrypted ads with encrypted content.

these files are served mainly for mobile devices(i see the same ads in the android app).
it's possible to distinguish between ads and content(unencrypted ads with encrypted content).
however it's better to extract the old manifests.
i think it's caused by a change in the site related to how it respond to requests from mobile devices https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/cbs.py#L56.

i have a fix for this but it involve making a lot of request to extract all formats.

The problem is CBS only serves the 720p stream to Android devices. I was able to take this m3u8 url:
http://once.unicornmedia.com/now/master/playlist/bb0b18ba-64f5-4b1b-a29f-0ac252f06b68/77a785f3-5188-4806-b788-0893a61634ed/93677179-2d99-4ef4-9e17-fe70d49abfbf/content.m3u8
and I found the 720p stream:
http://api016-phx.unicornmedia.com/now/media/playlist/bb0b18ba-64f5-4b1b-a29f-0ac252f06b68/77a785f3-5188-4806-b788-0893a61634ed/468fb310-a585-11e4-bfdb-005056837bc7/93677179-2d99-4ef4-9e17-fe70d49abfbf/0/0/2542/content.m3u8?visitguid=18c75a73-5452-4995-b5cf-0c0755432915&segmentlength=10&adsegmentlength=10&protocolversion=3
Like you said the commercials are included. However I just ctrl+f #EXT-X-KEY:METHOD=NONE and delete everything in between that and #EXT-X-DISCONTINUITY - then I saved the m3u8 file to my local hdd and ripped it with ffmpeg. Worked great.

 The commercial segments stick out like a sore thumb when you know what to look for in the m3u8.
The commercials all have #EXT-X-KEY:METHOD=NONE immediately above them. So if you start at the line that says #EXT-X-KEY:METHOD=NONE, delete that line and every subsequent line until you reach the next line that says #EXT-X-DISCONTINUITY (and delete that line as well). The next line after that is always the key file:
#EXT-X-KEY:METHOD=AES-128,URI="http://once.unicornmedia.com/key/77a785f3-5188-4806-b788-0893a61634ed/93677179-2d99-4ef4-9e17-fe70d49abfbf/bb0b18ba-64f5-4b1b-a29f-0ac252f06b68/once.key?umx=cAy9b5515NitZ2XV9RnbyA==",IV=0xba180bbbf5641b4ba29f0ac252f06b68
which can be deleted as well since you only need it once at the very beginning. Below that line it's all show segments like this:
#EXTINF:9.009,
http://api016-phx.unicornmedia.com/now/media/segment/bb0b18ba-64f5-4b1b-a29f-0ac252f06b68/77a785f3-5188-4806-b788-0893a61634ed/468fb310-a585-11e4-bfdb-005056837bc7/93677179-2d99-4ef4-9e17-fe70d49abfbf/0/0/270/content.ts?visitguid=18c75a73-5452-4995-b5cf-0c0755432915&baseguid=93677179-2d99-4ef4-9e17-fe70d49abfbf&streamDuration=3330&umx=cAy9b5515NitZ2XV9RnbyA==&startsegmentseconds=0&endsegmentseconds=10
The number of each segment goes up by 10 seconds, so the second segment is "&endsegmentseconds=20" - the third is "&endsegmentseconds=30" etc. It continues like that until you see another #EXT-X-DISCONTINUITY
Delete that and every line until the next key file (that line also gets deleted just like before). Rinse, lather, repeat until the end. Seems like something that could be done by a script in a fraction of a second.

The commercials all have #EXT-X-KEY:METHOD=NONE immediately above them.

not always(they can change it see Content Security -> AES-128 in http://docs.brightcove.com/en/once/guides/once-vod-2-0.html)

which can be deleted as well since you only need it once at the very beginning.

it's possible for the key or the IV to change.
the way to check if the segement is an or not is by checking the mediaItemId of the segement is the same as the mediaItemId of the m3u8 media playlist.
also i don't think that changing the m3u8 manifest is something to be done in youtube-dl.

Well you guys are a lot smarter than I am I look forward to see what solution you come up with :)
In the mean time I'll do it the hard way, lol...

My workaround is to use --write-pages, then edit the *.m3u8.dump file and change "adsegmentlength=10" to "adsegmentlength=0".
Since youtube-dl wont let me specify a "file:///" URL, I throw it up on a site I have access to and youtube-dl that without any issues.


Machine translation blocks.main_loop error

ran the machine translation and got this error at validation:
INFO:machine_translation.sampling:Started Validation:
ERROR:blocks.main_loop:Error occured during training.
Blocks will attempt to run on_error extensions, potentially saving data, before exiting and reraising the error. Note that the usual after_training extensions will not be run. The original error will be re-raised and also stored in the training log. Press CTRL + C to halt Blocks immediately.
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
...
...
...
Original exception:
UnusedInputError: theano.function was asked to create a function computing outputs given certain inputs, but the provided input variable at index 3 is not part of the computational graph needed to compute the outputs: sequencegenerator_generate_outputs.
To make this error into a warning, you can pass the parameter on_unused_input='warn' to theano.function. To disable it completely, use on_unused_input='ignore'.
Use of uninitialized value $length_reference in numeric eq (==) at ./data/multi-bleu.perl line 148.
blocks 0.1.1
fuel 0.1.1
bleeding edge Theano 0.7.0

Can you please add on_unused_input='warn' to your THEANO_FLAGS and try again.

orhanf thanks on_unused_input='warn' solved it for me

@rizar this issue can be closed, thanks.

@rizar  Is it reliable to use the flat  on_unused_input='warn' on this occasion?


after scan the statistics page still show zero files exclude

I have a lot files like node_modules can be scanned, but after scan the statistics page has nothing change.

by the way, I got some error log

same issues
intel，macos version: 14.1.2


Document configuration in the wiki

In #14 it was noted that in some cases you need to add context.PathBase to the UrlPath property. This, and other, configuration needs to be documented on the wiki

Updated documentation at https://github.com/smatsson/tusdotnet/wiki/Configure-tusdotnet


M3-4553 Fix tab-based navigation

Description
When we made the switch to reach tabs, the active tab was only reflected in the UI, not in the url (unless you hard-refresh). This fixes that as you switch tabs, either via click or with keyboard, the url will be updated. This is based off work Jared had done here: https://github.com/linode/manager/pull/6757/ and converts any tab components using TabLinkList to follow this pattern (as that component is using Link from react-router under the hood). The one section I'm a little hesitant on is the Linode Create flow- this section feels noticeably slower but we're also updating create type in redux so there is a bit more complexity going on here than elsewhere.
Type of Change

Bug fix ('repair')

Note to Reviewers
I looked into converting to reach-router instead, but this article  convinced me to not for 2 reasons:
• React Router will be the true 'surviving' project, though Reach will still be maintained via bug fixes.
• "If you have a lot of routes, especially with a lot of nested Route and Switch components, you may want to pick React Router. It will be 100% backward compatible, offering an incremental migration path to the new hook-based APIs."- I feel that this fits our uses more, since we use tabs/React-Router pretty extensively throughout the app and in more complex ways than I think Reach Router is designed to support.

Good question @DevDW ! Of course we can always go back and iterate, especially since that section could use more refinement, but for the scope of this ticket I would agree a discussion of tradeoffs is in order. What is most necessary for this section to have the best experience?

On Linode Create, the speed is not great, but what's blocking is that you can't tab through the creation modes. When you arrow to a new tab, the focus changes to the H1 (I think).
Fundamentally what's happening with all of this is that by changing browser history, we're re-rendering the page, even though lots of the content is outside of the tabbed area. This happens everywhere, but Linode Create is huge so the cost is high. I'm not sure there's any way around this. we rely on url-based tab navigation for lots of stuff, but I don't know if we can have it both ways

Thanks for the feedback, folks! I'm going to remove these changes from Linode Create to gain our perceived performance back, as well as fix the callout Jared mentioned about focus being moved to headings within that section.


Support for Konsole and Termux

Theme for Konsole can also be uploaded to KDE Store, so it would be discoverable/installable directly through Konsole's options.

My apologies, @kirillmukhin Idk why I took so long to look at this.
As I do not have any good way to test this, do you mind sharing screenshots on how it looks? The file changes look correct.
Also, would you like to take credit and upload it to the KDE store? I can do it if you'd prefer.

No worries!
Here are the screenshots:
Konsole:

Termux:

I think I'd prefer if you would upload the Konsole theme.
To do so, you'll need to add product through Pling and select category App Addons>KDE App-Addons>Konsole Color Schemes.


New wraps: nonstd-{any,byte,expected,observer-ptr,optional,span,status-value,string-view}

Upstream projects:

https://github.com/martinmoene/any-lite
https://github.com/martinmoene/byte-lite
https://github.com/martinmoene/expected-lite
https://github.com/martinmoene/observer-ptr-lite
https://github.com/martinmoene/optional-lite
https://github.com/martinmoene/span-lite
https://github.com/martinmoene/status-value-lite
https://github.com/martinmoene/string-view-lite

These are individual future compatability headers for C++.
Having a nonstd-<header>-lite wrap would be really useful for these projects.

It seems every package is named like
set( unit_name       "any" )
set( package_name    "${unit_name}-lite" )

Each individual name looks unique enough so I am happy to create

https://github.com/mesonbuild/nonstd-any-lite
https://github.com/mesonbuild/nonstd-byte-lite
https://github.com/mesonbuild/nonstd-expected-lite
https://github.com/mesonbuild/nonstd-observer-ptr-lite
https://github.com/mesonbuild/nonstd-optional-lite
https://github.com/mesonbuild/nonstd-span-lite
https://github.com/mesonbuild/nonstd-status-value-lite
https://github.com/mesonbuild/nonstd-string-view-lite


Add more information about changes when opting-in to MSTest SDK

Summary
Relatest to https://github.com/microsoft/testfx/issues/3205

Internal previews

📄 File
🔗 Preview link

docs/core/testing/unit-testing-mstest-sdk.md
MSTest SDK overview

Ping @IEvangelist


OpenSearch 2.4.0 is available today!

Describe the blog post your would like to write
A clear and concise description of what subject the blog post will cover.
Blog post will announce the availability of OpenSearch 2.4.0
What is the title of the blog post?
Tentatively "OpenSearch 2.4.0 is available today!"
Who are the authors?
Who will be listed as the authors of this blog post
James McIntyre and others TBC
What is the proposed posting date?
Provide the date which you wish this post to go live
Nov. 10, 2022

https://github.com/opensearch-project/project-website/pull/1159


Teaser image 404

https://wechaty.js.org/2020/12/25/python-wechaty-room-inviter-plugin/ seems that the url of the teaser image is not right.
Need to be fixed.
And for answering how to get a  Wechaty Puppet Service Token from our providers, we need to prepare to publish an entry from our documentation website, instead of linking to outside.

Fixed.

Fixed.


DatePicker Calendar year is showing additional up and down arrows

Detailed description

Describe in detail the issue you're having.

Is this issue related to a specific component?
DatePicker

What did you expect to happen? What happened instead? What would you like to see changed?
I expected to see only blue up and down arrows instead I am seeing additional arrows on top of year element.

What browser are you working in?

Tried on MacOS Mojave 1. Chrome 2. Firefox (Both browsers with the latest version)

What version of the Carbon Design System are you using?
"carbon-components-react": "^6.99.1",
"carbon-components": "^9.69.0",

What offering/product do you work on? Any pressing ship or release dates we should be aware of?
Working on IBM internal project.

Steps to reproduce the issue
Add DatePicker range component and check the calendar.

Please create a reduced test case in CodeSandbox
https://codesandbox.io/s/github/IBM/carbon-components-react/tree/master/examples/codesandbox

Additional information

Screenshots or code

Attached above.
Code screenshot is here: (Code generated by Carbon)

Notes

This comes from using an input with type: number. There’s no official way of styling it away. We should instead use an input with type of text and then use the pattern attribute to check.
I think this input is generated by the flatpickr library.

Thanks @vpicone for finding this out! Let's do it: #2004

Yes upgrading carbon to latest version fixed this problem. Thank you.


Disallow vehicle spawning when in vehicle

I wrote some code before that was removed later (I guess @GTANAdam forgot to put it back when he cleaned up that code): https://github.com/GTANetworkDev/ExampleResources/commit/ba5065a41a7e928f98c7034dc939150fee1021b3#diff-a3e1d095eeaed2552a6c2f2654aa47f3L388
That basically deletes the current vehicle and spawns the player inside of the new vehicle. I think that's a much better solution that what you're proposing here :) What do you think?

Actually the code is there,

Handle(sender).VehicleHistory.Add(veh);
if (Handle(sender).VehicleHistory.Count > 2)
{
API.deleteEntity(Handle(sender).VehicleHistory[0]);
Handle(sender).VehicleHistory.RemoveAt(0);
}
    API.setPlayerIntoVehicle(sender, veh, -1);

Though i think it should be limited to 1 car, not 2.

No, that code is for limiting the amount of vehicles you can spawn. I'm talking about deleting the current vehicle you're in when spawning another one even when you're not reaching the limit.


[SPARK-21428][SQL][FOLLOWUP]CliSessionState should point to the actual metastore not a dummy one

What changes were proposed in this pull request?
While running bin/spark-sql, we will reuse cliSessionState, but the Hive configurations generated here just points to a dummy meta store which actually should be the real one. And the warehouse is determined later in SharedState, HiveClient should respect this config changing in this case too.
How was this patch tested?
existing ut
cc @cloud-fan @jiangxb1987

Can one of the admins verify this patch?

@gatorsmile Hi Sean,
I am hitting this issue. Actually this seems like a regression as my script which was working before is no longer working. Here is my scenario.

spark-sql
create database testdb;
exit spark-sql
spark-sql
use testdb;  => I get a database not found error.
Can we please have this reviewed and merged if the changes are good. Thanks in advance.

@dilipbiswal This is because the CliSessionState instance initialized in SparkCLISQLDriver pointing to a dummy metastore and reused later in the hive metastore client

cc @cloud-fan @jiangxb1987

cc @cloud-fan @jiangxb1987 @gatorsmile pr title and descriptions updated

OK to test

@dilipbiswal does your script work after this PR?

@cloud-fan Yeah.. I have tried my script against this PR and it works fine. I am not familiar with the changes and don't know if it can have any side effects.  One thing that haven't had the time to find out is in my script ..
1) spark-sql
    create database testdb;
2) exit spark-sql
3) spark-sql
    use testdb;  => I get a database not found error.

How did the create database succeed i.e i didn't get any error ? If it did, then where did it create the database at ? Perhaps you know the answer :-)

@cloud-fan it didn't trigger the test ?

@cloud-fan  The cliSessionState is meant to be reused but discarded for isolated hive client classloader couldn't get it through SessionState.get(), so hive client will generated a SessionState instance everytime while calling HiveClient.newSession(). In my foregoing pr, I made it reused but i didn't notice that it is just pointing to a dummy metatstore. This has to be fixed.

jenkins unreachable cc @cloud-fan

ok to test

Test build #81718 has started for PR 19068 at commit 267a1b2.

Test build #81721 has started for PR 19068 at commit 9682eab.

Test build #81718 has finished for PR 19068 at commit 267a1b2.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81718/
Test FAILed.

Test build #81721 has finished for PR 19068 at commit 9682eab.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81721/
Test FAILed.

can you run ./build/sbt "hive/test" locally to make sure all hive tests pass?

Test build #81906 has started for PR 19068 at commit b21fc72.

Test build #81906 has finished for PR 19068 at commit b21fc72.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81906/
Test FAILed.

Merged build finished. Test FAILed.

Test build #81909 has started for PR 19068 at commit f2618b9.

@cloud-fan i met linkage err before, and now i simplify the logic, could you trigger jenkins before reverting

Test build #81909 has finished for PR 19068 at commit f2618b9.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81909/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Test build #81910 has started for PR 19068 at commit c5c1c26.

Test build #81910 has finished for PR 19068 at commit c5c1c26.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81910/
Test FAILed.

retest this please

retest this please

Test build #81916 has started for PR 19068 at commit c5c1c26.

Test build #81916 has finished for PR 19068 at commit c5c1c26.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81916/
Test PASSed.

LGTM, merging to master!

@yaooqinn
This change works well for me, thanks for fix !
After this change, hive client for execution(points to a dummy local metastore) will never be used when running sql inspark-sql, hive client points a true metastore. Right ?
So why it is designed to have the hive client in SparkSQLCLIDriver points to a dummy local metastore before ? Is this change breaking some design?


Israeli calendar

This isn't trivial since the Israeli holidays (consisting of mainly Jewish holidays) do not belong to the Georgian calendar, but rather the Hebrew calendar.
I think a package such as pyluach might be of use here, manipulating Hebrew calendar dates and making Hebrew-Georgian calendar conversions.
docs:

https://en.wikipedia.org/wiki/Public_holidays_in_Israel
https://en.wikipedia.org/wiki/Hebrew_calendar

Indeed! far from trivial. Last time I've checked, I only found an obscure library that was only available from archive.org, which was far from usable, and I'm not talkin about Python 3 compatibility...
The library you're pointing at looks interesting, and fully compatible with workalendar specs (Python 2.7 -> 3.6). That's a good start. I'll add this to the backlog.

I might be working on it next week, I will send a PR. Looking at the code I had two questions:

What is the correct approach to holidays which are optional paid leave or unpaid leave?
Will it be appropriate to implement the holidays which are based on the Jewish calender using get_variable_days?

Hey @armona! Just so you know, I've already built a skeleton for the Israeli calendar right here. I forked the repo, added a branch 'israel' and added the calendar to every necessary place as mentioned in the contributing doc. Moreover, I made tests for all of the holidays in 2017 and 2018.
You can see it all in the commit. So it's only left to code the calendar itself. :smiley:
Please let me know if and when you work on it, I'd love to be of help if possible!


orchestrator-client support for headers auth

Hi @shlomi-noach,
I would like to add support for headers auth for orchestrator-client script. What do you think about it?
Thanks, Max

PR #972


Java 8 Stream Documentation improvements

Clarifies the use of Streams for both eager and lazy evaluation uses.

@mariuszs
Thanks for all your input.


encodedSemanticClassifications-full should be in the public tsserver protocol API

Bug Report
The encodedSemanticClassifications-full request is marked as @internal [1] and not exposed in the ts.server.protocol namespace. Yet the typescript-language-features in VSCode [2] and another LSP implementation [3] of tsserver API uses it for providing semantic highlighting.
[1] https://github.com/microsoft/TypeScript/blob/dbeae5d943c784661862c52b8e215a2907c31a33/src/server/protocol.ts#L102-L103
[2] https://github.com/microsoft/vscode/blob/1900abc88b8a0bce7446314c248c5e79deaad0c1/extensions/typescript-language-features/src/languageFeatures/semanticTokens.ts#L66-L66
[3] https://github.com/typescript-language-server/typescript-language-server/blob/3722b51c0ad8e758c4e42f622bbe25ae981071e1/src/lsp-server.ts#L1254-L1254
🔎 Search Terms

🕗 Version & Regression Information

⏯ Playground Link

💻 Code

🙁 Actual behavior
The encodedSemanticClassifications-full request type is not considered a public API.
🙂 Expected behavior
Should be considered a public API.

My suggestion would be to call it experimental then.
But regardless of the naming, the main point of this issue is that this comment prevents it from being exposed in the public tsserver protocol.


mfrc522.PICC_IsNewCardPresent() function

Hi,
I'm trying to use this function to get an output if a card is present on the reader or not. In my application I need to keep cards on the reader, and the reader to constantly spit out the card ID (once every second or so), and at the same time a boolean marker if any card is present or not.
But I can't get it to work. Any pointers?
Thanks.
edit:
I've tried this code:
if ( ! mfrc522.PICC_IsNewCardPresent())          {           test=1;               Serial.println(test);              delay(50);              return;           }       else {         test=0;         Serial.println(test);         }
But the output is "1" when no card is present, and if a card is present it alternates between "0" and "1".

I really don't understand what you are trying to do...
However, the samples use some convenience functions such as dumping the card info. Thos methods call HaltA so there is no need to call it again. Your code is incomplete but reading the results you obtain I guess you ar not calling HaltA when you get the ATQA (Which is requested in the convenience function PICC_IsNewCardPresent Yo could try something like this:
void loop() {
  // Look for new cards
  if ( ! mfrc522.PICC_IsNewCardPresent()) {
    Serial.println("No card");
    return;
  }

  Serial.println("Present");
  mfrc522.PICC_HaltA();
}

I am trying to get a boolean marker that says wether a tag is present or not at all times. Thanks for the info, I'll check out your code.
What I am using right now is this:
 void loop() {   if ( ! mfrc522.PICC_IsNewCardPresent()) {     delay(50);     return;   }   if ( ! mfrc522.PICC_ReadCardSerial()) {     delay(50);     return;   }
It works for spitting out the tag. The code in the first post I used trying to add a marker for when any tag is present.

I am trying to get a boolean marker that says wether a tag is present or not at all times. Thanks for the info, I'll check out your code.
What I am using right now is this:
void loop() { if ( ! mfrc522.PICC_IsNewCardPresent()) { delay(50); return; } if ( ! mfrc522.PICC_ReadCardSerial()) { delay(50); return; }
It works for spitting out the tag. The code in the first post I used trying to add a marker for when any tag is present.

I used your code and indeed I got No card or Present on serial.
What I am trying to do, is get a Present and another action. I'm sending the code to a file on a local network server. I can't seem to make it do another thing along with Present in the loop.
I am using this code:
#include  "MFRC522.h"
#include "SPI.h"
#define RST_PIN 5
#define SS_PIN  4
MFRC522 mfrc522(SS_PIN, RST_PIN);
#include <ESP8266WiFi.h>
#include "lwip/opt.h"
void setup() {
  Serial.begin(115200);
  SPI.begin();
  mfrc522.PCD_Init();
  delay(10);
  }

void loop() {
  if ( ! mfrc522.PICC_IsNewCardPresent()) {
    Serial.println("No card");
   // delay(500);
    return;
  }
  if ( ! mfrc522.PICC_ReadCardSerial()) {
    delay(500);
    return;
  }
  dump_byte_array(mfrc522.uid.uidByte, mfrc522.uid.size);
  Serial.println("Present");
  mfrc522.PICC_HaltA();
}
void dump_byte_array(byte *buffer, byte bufferSize) {
  for (byte i = 0; i < bufferSize; i++) {
    Serial.print(buffer[i] < 0x10 ? " 0" : " ");
    Serial.print(buffer[i], HEX);
  }
}

I tried it in all combinations and can't make it print the card number consistently. In the actual form, whenever I scan a card I get the card's serial number followed by "Present" but the goes to "No card" continuously even if the card is on the reader. I have to take it off and put it back again to get another serial read.

Strangest thing. I was having trouble with your code as well. If I cycle the reader without a card in the proximity, all I get is No card, even if I get the card close to the reader.
But, if I start the reader with the card on it, it starts with "Present" and then it works as it should. I think this threw me off. I need to start again using your code.
But why would that happen? How can I make it work normally?

If so I might have a solution, but this solution involves constantly
reseting the driver. If this helps I can provide an example :)
On 26 January 2017 at 10:03, bob808<EMAIL_ADDRESS>wrote:

I am trying to get a boolean marker that says wether a tag is present or
not at all times. Thanks for the info, I'll check out your code.
What I am using right now is this:
void loop() { if ( ! mfrc522.PICC_IsNewCardPresent()) { delay(50);
return; } if ( ! mfrc522.PICC_ReadCardSerial()) { delay(50); return; }
It works for spitting out the tag. The code in the first post I used
trying to add a marker for when any tag is present.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/miguelbalboa/rfid/issues/279#issuecomment-275427659,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKoVjLHHMBtNM904cWiXiyyTa5KqDXaEks5rWMPigaJpZM4Ltv7m
.

--
Un cordial saludo.
Moises Cedeño Torres

I'm trying to see if there's a better way :) I need to update a file on my server constantly with the present tag code, and check if it's present or not.

I could use a magnet and a hall effect sensor for presence detecting. Would be a cleaner way than resetting. But still messy.

Ok BOb, let us know if you find a clean way to do that, if not and still
need the dirty eway let me know ;)
On 27 January 2017 at 13:47, bob808<EMAIL_ADDRESS>wrote:

I could use a magnet and a hall effect sensor for presence detecting.
Would be a cleaner way than resetting. But still messy.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/miguelbalboa/rfid/issues/279#issuecomment-275757033,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKoVjJ3IQTEDfqlB6gqdkuiACRX9oVfUks5rWknegaJpZM4Ltv7m
.

--
Un cordial saludo.
Moises Cedeño Torres

Ok mate, thanks!
So far I think I'm going to go with hall effect sensors but I'd love to have a way to only use the rfid reader.

i am using this

// Look for cards
 if ( ! mfrc522.PICC_IsNewCardPresent())
 {
    delay(50);
  if ( ! mfrc522.PICC_IsNewCardPresent())
  {
    delay(50);

    // no card detect
    return;
    }
  }

 //card detect

@bob808 Can this be closed?

@simbadMarino @Rotzbua Can you please provide me with code that read the RFID card continuously  PLEASE PLEASE

I am using RFID multi read code `#include <SPI.h>
#include <MFRC522.h>
constexpr uint8_t RST_PIN = D0;     // Configurable, see typical pin layout above
constexpr uint8_t SS_1_PIN = D1;   // Configurable, take a unused pin, only HIGH/LOW required, must be diffrent to SS 1
constexpr uint8_t SS_2_PIN = D2;    // Configurable, take a unused pin, only HIGH/LOW required, must be diffrent to SS 2
constexpr uint8_t SS_3_PIN = D3;    // Configurable, take a unused pin, only HIGH/LOW required, must be diffrent to SS 3
String y[] = {"A0A1925E", "D0589D5E", "D0A8BEA3"};
String x[3];
constexpr uint8_t NR_OF_READERS = 3;
int state = 0;
byte ssPins[] = {SS_1_PIN, SS_2_PIN, SS_3_PIN};
MFRC522 mfrc522[NR_OF_READERS];   // Create MFRC522 instance.
/**
Initialize.
*/
void setup() {
Serial.begin(115200); // Initialize serial communications with the PC
while (!Serial);    // Do nothing if no serial port is opened (added for Arduinos based on ATMEGA32U4)
SPI.begin();        // Init SPI bus
for (uint8_t reader = 0; reader < NR_OF_READERS; reader++) {
mfrc522[reader].PCD_Init(ssPins[reader], RST_PIN); // Init each MFRC522 card
Serial.print(F("Reader "));
Serial.print(reader);
Serial.print(F(": "));
mfrc522[reader].PCD_DumpVersionToSerial();
mfrc522[reader].PCD_SetAntennaGain(mfrc522[reader].RxGain_max);
}
}
/**

Main loop.
*/
void loop() {

for (uint8_t reader = 0; reader < NR_OF_READERS; reader++) {
// Look for new cards
if (mfrc522[reader].PICC_IsNewCardPresent() && mfrc522[reader].PICC_ReadCardSerial()) {
  Serial.print(F("Reader "));
  Serial.print(reader);
  // Show some details of the PICC (that is: the tag/card)
  Serial.print(F(": Card UID:"));
  dump_byte_array(mfrc522[reader].uid.uidByte, mfrc522[reader].uid.size);
  Serial.println();
  Serial.print(F("PICC type: "));
  MFRC522::PICC_Type piccType = mfrc522[reader].PICC_GetType(mfrc522[reader].uid.sak);
  Serial.println(mfrc522[reader].PICC_GetTypeName(piccType));

  // Halt PICC
  mfrc522[reader].PICC_HaltA();
  // Stop encryption on PCD
  mfrc522[reader].PCD_StopCrypto1();
} //if (mfrc522[reader].PICC_IsNewC

} //for(uint8_t reader
}
/**

Helper routine to dump a byte array as hex values to Serial.
*/
void dump_byte_array(byte *buffer, byte bufferSize) {
for (byte i = 0; i < bufferSize; i++) {
Serial.print(buffer[i] < 0x10 ? " 0" : " ");
Serial.print(buffer[i], HEX);
}
}`

Hi. I had a similar problem and I solved it very simply.
if (!mfrc522.PICC_IsNewCardPresent()) {     delay(50);     if(!mfrc522.PICC_IsNewCardPresent())       return false;   }   if (!mfrc522.PICC_ReadCardSerial()) {     delay(50);     if(!mfrc522.PICC_ReadCardSerial())       return false;   }

I realize that you might need a more complicated solution for many applications, but if you're looking for a really quick and easy way to test out continuous tag reading, here's one solution:
Just start with the ReadNUID example from the MFRC522 library, and comment out the following lines:
`  // Halt PICC
//rfid.PICC_HaltA();
// Stop encryption on PCD
//rfid.PCD_StopCrypto1();`
If you're interested in running a really simple version of the ReadNUID example, with some extraneous features stripped out, I'm including the full file below. All I did was comment out parts of the original example -- no additions. I found this was a good way to check that my hardware was working correctly, without adding additional complications from the software.
`
#include <SPI.h>
#include <MFRC522.h>
#define SS_PIN 10
#define RST_PIN 9
MFRC522 rfid(SS_PIN, RST_PIN); // Instance of the class
//MFRC522::MIFARE_Key key;
// Init array that will store new NUID
//byte nuidPICC[4];
void setup() {
Serial.begin(9600);
SPI.begin(); // Init SPI bus
rfid.PCD_Init(); // Init MFRC522
/*  for (byte i = 0; i < 6; i++) {
key.keyByte[i] = 0xFF;
}
Serial.println(F("This code scan the MIFARE Classsic NUID."));
Serial.print(F("Using the following key:"));
printHex(key.keyByte, MFRC522::MF_KEY_SIZE); */
}
void loop() {
// Look for new cards
if ( ! rfid.PICC_IsNewCardPresent())
return;
// Verify if the NUID has been readed
if ( ! rfid.PICC_ReadCardSerial())
return;
Serial.print(F("PICC type: "));
MFRC522::PICC_Type piccType = rfid.PICC_GetType(rfid.uid.sak);
Serial.println(rfid.PICC_GetTypeName(piccType));
// Check is the PICC of Classic MIFARE type
if (piccType != MFRC522::PICC_TYPE_MIFARE_MINI &&
piccType != MFRC522::PICC_TYPE_MIFARE_1K &&
piccType != MFRC522::PICC_TYPE_MIFARE_4K) {
Serial.println(F("Your tag is not of type MIFARE Classic."));
return;
}
/*
if (rfid.uid.uidByte[0] != nuidPICC[0] ||
rfid.uid.uidByte[1] != nuidPICC[1] ||
rfid.uid.uidByte[2] != nuidPICC[2] ||
rfid.uid.uidByte[3] != nuidPICC[3] ) {
Serial.println(F("A new card has been detected."));
// Store NUID into nuidPICC array
for (byte i = 0; i < 4; i++) {
  nuidPICC[i] = rfid.uid.uidByte[i];
}

*/
Serial.println(F("The NUID tag is:"));
Serial.print(F("In hex: "));
printHex(rfid.uid.uidByte, rfid.uid.size);
Serial.println();
Serial.print(F("In dec: "));
printDec(rfid.uid.uidByte, rfid.uid.size);
Serial.println();
//}
//else Serial.println(F("Card read previously."));
// Halt PICC
// rfid.PICC_HaltA();
// Stop encryption on PCD
//rfid.PCD_StopCrypto1();
}
/**

Helper routine to dump a byte array as hex values to Serial.
*/
void printHex(byte *buffer, byte bufferSize) {
for (byte i = 0; i < bufferSize; i++) {
Serial.print(buffer[i] < 0x10 ? " 0" : " ");
Serial.print(buffer[i], HEX);
}
}

/**

Helper routine to dump a byte array as dec values to Serial.
*/
void printDec(byte *buffer, byte bufferSize) {
for (byte i = 0; i < bufferSize; i++) {
Serial.print(buffer[i] < 0x10 ? " 0" : " ");
Serial.print(buffer[i], DEC);
}
}`

My understanding is UID is read only the 1st time a card is detected using PICC_IsNewCardPresent
See #437 for more details

Was this question ever resolved? I'm trying to get a motor to run while a the PICC is present on the sensor. From what I've read here and on other forums it looks like this is not possible without extra hardware. Is my understanding correct?
I have the same issue as @bob808 that when the card is present, the output from mfrc522.PICC_IsNewCardPresent() alternates between 1 and 0.

Alternating seems to be a feature of  the library or hardware
so we may have to live with it.
My suggestion.
flag=0;
if (mfrc522.PICC_IsNewCardPresent())
{
flag=1; // card present
}
else  //try again to catch the toggle effect
{
if (mfrc522.PICC_IsNewCardPresent())
flag=1; // card present
}
if (flag)
{
Serial.println("Card present");
}
else
{
Serial.println("No card");
}
delay(500);

Similar to @ans27g suggestion, you can simply also do something like:
if (!mfrc522.PICC_IsNewCardPresent() && !mfrc522.PICC_IsNewCardPresent()) {
    Serial.println("PICC_IsNewCardPresent ---> Hell NO!");
    return;
} else {
    Serial.println("PICC_IsNewCardPresent ---> YES!");
}


Menu: move "User Manager" to "User"

I'm totally unsure this is a good idea. The "user" section was added by us to make room for the user-specific password changer, which was almost completely unreachable in the previous navigation structure. Then we added the logout, a feature associated to the actual user/session.
The user manager is fundamentally different as it is a admin-oriented task that makes sense to be under "system".
System modifies all aspects of a system, user only modifies things for a particular user. Does that sound reasonable?

Yes, should I close this issue?

Just did not want to close it prematurely. Will so now, thanks.


Heading references are not counted within the same file

Just updated from 1.2.5 to 2.1.4, noticed that heading references within the same file are not counted. Heading references from other files seem to be counted fine, though I didn't do any thorough testing.
1.2.5

2.1.4

I came here to report the same problem.

This should now be addressed with update 2.2.0. Thank you for reporting this issue.


change request uri in GatewayFilter has no effect

Describe the bug
Please provide details of the problem, including the version of Spring Cloud that you
are using.
ServerHttpRequest request = exchange.getRequest().mutate().uri(newUri).build();
exchange.mutate().request(request).build();
Sample
If possible, please provide a test case or sample application that reproduces
the problem. This makes it much easier for us to diagnose the problem and to verify that
we have fixed it.

    ServerHttpRequest request = exchange.getRequest().mutate().uri(newUri).build();
    exchange.mutate().request(request).build();

this code of change request uri is not working, it doesn't have effect

@chengtangzheng2021
You code just create new ServerWebExchange object

thanks ctlove0523, made it working now

Can you please tell how it worked. I too face same problem
ServerHttpRequest request = exchange.getRequest().mutate().uri(newUri).build();
exchange.mutate().request(request).build();
is not updating URI.

Can you please tell how it worked. I too face same problem ServerHttpRequest request = exchange.getRequest().mutate().uri(newUri).build(); exchange.mutate().request(request).build(); is not updating URI.

Did you solve


TypeError: RandomResizedCrop.call() missing 1 required positional argument: 'label'

Hello, I encountered the following error while running the code:
Traceback (most recent call last):
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/trainer.py", line 215, in <module>
    main()
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/trainer.py", line 211, in main
    run(0, 1, args)
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/trainer.py", line 146, in run
    model, optimizer, loss_info = train(model, train_loader, loss_fn, optimizer, device, local_rank, nprocs)
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/train.py", line 28, in train
    for image, target_points, target_density in data_iter:
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 634, in __next__
    data = self._next_data()
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 678, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/datasets/crowd.py", line 171, in __getitem__
    images_labels = [self.transforms(image.clone(), label.clone()) for _ in range(self.num_crops)]
  File "/mnt/C/CrowdCounting/CLIP-EBC-main/datasets/crowd.py", line 171, in <listcomp>
    images_labels = [self.transforms(image.clone(), label.clone()) for _ in range(self.num_crops)]
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/mnt/A/.conda/envs/CLIP-EBC-main/lib/python3.10/site-packages/torchvision/transforms/v2/_container.py", line 51, in forward
    sample = transform(sample)
TypeError: RandomResizedCrop.__call__() missing 1 required positional argument: 'label'

Has anyone encountered the same problem? Are there any solutions available?

Hi @3555367879 ,
I have tested the code again, but I am not seeing this error. Could you please double check whether you are using the transform functions defined in datasets.transforms.py? Please do not use any transforms officially implemented by PyTorch, as they don't correspondingly transform the dot annotations.
Looking forward to your reply.
Many thanks,
Yiming

Hi @3555367879 ,
Thanks for letting me know. Since this issue has been resolved, I am going to close it. But feel free to create GitHub issues when you see other problems.
Many thanks,
Yiming


Programatically accessing the HTML Report

We have a use case where it would be useful to access the HTML report from within the Rust code itself - we would like to upload it to S3 from inside a regular Fargate task each time it runs. It would be nice to use the AWS SDK to do this where we can generate the report and then when the load test is done, run the code to upload to S3.
The alternative is to chain some scripts together inside docker which is doable but not as nice.
Is there already a way to access this kiond of functionality, or is it something that needs to be done through an alternative approach?

Goose will write logs and reports locally. If you want it to write directly to S3, you'd have to mount S3 on a local mountpoint and then have Goose wrirte there. What we typically do, however, is use a simple shell-script wrapper which copies any/all artifacts to whatever desired permanent storage when the load test finishes.

Grand, I think that's the approach we'll take then. I just wanted to make sure there wasn't anything in the library itself as well. Thanks!


Rabit should import dmlc-core as a CMake target

Similar to #82 , I'm testing PR #113 and in order to compile rabit I need to pull in the dmlc-core headers dmlc-core/include/dmlc and copy those into rabit/include to make the project compile.
First: Am I doing this right?
Second: If the dmlc headers are indeed a requirement to the project, should they be included (and version) in the rabit repo? Not sure what the motivation is for keeping them separate (I can imagine downstream projects that depend on both like xgboost though)

Indeed, I recently found a subtle bug (#5514) because some flags from dmlc-core didn't get propagated to Rabit. Rabit should import dmlc-core as a proper CMake target.

Rabit is now part of XGBoost.