Prisma client can't reach database server from time to time

Bug description
Hi,
We are running a very classic NodeJS Express API connected to a PostgreSQL database via the Prisma client, and from time to time we have requests failing with the error P1001 :
Can't reach database server at ...
Please make sure your database server is running at ...

And everything is ok in the previous or following requests few milliseconds apart:

I checked on the infrastructure side, and the database seems reachable from the server with no issues of connection pool,  overload, high CPU, high memory or whatever else.
Is there something I can check or try to solve that ?
How to reproduce
I didn't find any way to reproduce it constantly but it seems to happen a little bit more during low traffic periods like night. And it seems that sending queries every 20 seconds reduce a lot the issue. So it's maybe linked to idle timeout or something similar.
Expected behavior
No more requests failing with this error of course 😅 !
I don't expect a fix or whatever, but more at advices or things I can take a look to to move forward in solving this problem.
Prisma information
It happens with different queries, we even have very basic health checks failing from time to time due to this error:
export default async function () {
  await client.$queryRaw`SELECT 0`;
}

Environment & setup

OS: Ubuntu
Database: PostgreSQL
Node.js version: 16.13.0

Prisma Version
3.10.0

Do you possibly have database side logging for that time and can see if the requests really reach the database?
Could it potentially be a temporary network thing that affects only some connection attempts?
You could increase the timeout and see if that changes something. Potentially slightly slower requests are preferrable to these errors. Alternatively you could add retries for these queries with this error message. Does the second attempt work then?

Do you possibly have database side logging for that time and can see if the requests really reach the database?
I can't see specifically if these requests reached the database or not, but I see that there is a lot of disconnection / connection in one minute between the apps ands the database (I don't know if it's an expected behavior or not ):

Could it potentially be a temporary network thing that affects only some connection attempts?
I don't think so, I have another very similar app (a NodeJS Express API) with the same infrastructure but which use Sequelize and I don't have this kind of errors.

You could increase the timeout and see if that changes something.
Sounds interesting, how can I do that ?

Also, we are now running a very basic health check every minute:
export default async function () {
  await client.$queryRaw`SELECT 0`;
}

And it's seems to fix the issue, but it's more a dirty patch, I would like to know and fix the root cause.
The connection should remain open, even if there is no request for 30 seconds, no ?
I see that a lot of pg_conf settings like idle_session_timeout, tcp_keepalives_idle, tcp_keepalives_interval or idle_in_transaction_session_timeout exist, is it something I should look into or the default values are ok ?

I'm seeing similar behavior across 4 services. They are node services talking to an RDS aurora instance. It seems to happen equally across those service. I am also using prisma in other node services that talk to an RDS postgres instance and I haven't seen it happen there, though the sample size isn't huge. On the Aurora side my CPU, memory nor connections never spike to anywhere near the limits either.
It seems to happen using both prisma.<model>.findMany(...) and prisma.$queryRaw(...) for me. I'm also running inot the issue wiht various versions of prisma across those services: 3.1.1, 3.5.0 and 3.7.0.

@janpio I see in the Prisma doc there is this kind of settings, is it something I should look into or the default values are ok ?

is it something I should look into or the default values are ok ?

@Dammmien The Prisma chosen defaults are usually fine, you can increase the connect_timeout for example to see if the connection sometimes just needs a bit longer to connect.

And if you can tell me where is manage the connection / reconnection logic in the client source code, I can take a look.

There is none, that is something that would need to be implemented in your application or via a Prisma middleware.

It seems to happen equally across those services.

@drewcorlin1 At the same time or just over time?

I am also using prisma in other node services that talk to an RDS postgres instance and I haven't seen it happen there

Can you pinpoint what is different in these services vs. the one where it does happen and fail?

@janpio Over time its relatively equal per service. At the times that the issue has cropped up it seems to happen in groups, with about half of those groups being confined to a single service and half of them being spread across multiple services. I guess this may indicate that it's on the RDS side?
The only consistent thing that is different between these services seems to be Aurora vs Postgres.
Is there any extra logging that I'm able to add/enable in the prisma client? I'm adding better logging + monitoring on the RDS side and just want to try to cover all my bases

At the times that the issue has cropped up it seems to happen in groups, with about half of those groups being confined to a single service and half of them being spread across multiple services. I guess this may indicate that it's on the RDS side?

It at least indicates a correlation across different Prisma Client deployments that have no communication channel - so it might be network or RDS side indeed.
There is unfortunately no additional possible logging. The error message is thrown when the timeout is not enough to create a connection. Prisma does not have further information why that is not the case. (Possible reasons of course are: Local load, network, database side, phase of the moon.)

@janpio when I:

turn off my database locally
run a query (it fails)
turn on my database locally
run a query (it succeeds)

So there is a reconnection logic somewhere...
Also I tried to reconnect manually and retry the query in a middleware:
prisma.$use(async (params, next) => {
  try {
    const result = await next(params);

    return result;
  } catch (error) {
    await prisma.$disconnect();
    await prisma.$connect();
    const result = await next(params);
    return result;
  }
});

But the client is completely fucked up and never reconnect if the prisma.$connect(); fails again ...
Also I noticed that if the first prisma.$connect fails, the client never reconnect
No matter if I call it explicitly or with the first request

Ok, I misunderstand your ask for "retry" logic then - which we do not have. If a connection in unhealthy, it should be removed from the connection pool and then the next query or $connect would request a new one.  (The code sits in prisma/quaint btw and is not something you should ever have to look at)
($disconnect and $connect should actually completely restart Prisma and then get a completely empty connection pool - so what you are describing is pretty weird from that perspective.)

@janpio Thanks for the info! Just want to confirm 1 thing:

The error message is thrown when the timeout is not enough to create a connection

This is explicitly when trying to establish a new connection to the DB, not when attempting to get a connection from Prisma's pool right?

@janpio by taking a look into the code, it seems to be linked to this.libraryStartingPromise which is not set to undefined in the catch, so it do not try to reconnect:
https://github.com/prisma/prisma/blob/757a9c44b41a16482eab7b26ed5d14f123ea771d/packages/engine-core/src/library/LibraryEngine.ts#L324-L355

Nice, there are other issues and PRs about something like this (https://github.com/prisma/prisma/pull/12066 + https://github.com/prisma/prisma/issues/12028)
But is that related to your original issue here?

This is explicitly when trying to establish a new connection to the DB, not when attempting to get a connection from Prisma's pool right?

The way you ask makes me scared to answer @drewcorlin1 😆 But yes, fundamentally you should get a different answer for the connection pool, a connection pool timeout or other message that mentions the connection pool.

But is that related to your original issue here?
Not exactly, but with this fix, in case of error I could try to disconnect / reconnect the client and retry the query.
Without the fix if I try to reconnect and it fails the client is completely stuck

Hi, https://github.com/prisma/prisma/pull/12066 is merged and should hopefully fix this issue. Would you be able to test with prisma@dev and see if your issue is resolved.

Hey @Dammmien, Did you get a chance to test this out again? The fix should be now in the latest version.

Hi @pantharshit00 ,
Yes I noticed there was several fixes / improvements of the connection in the last Prisma client versions, many thanks 🙏🏻 !
We also did some changes on our sides: opened support ticket to our hosting provider, add a health check to the connection between our API and the databases, upgraded to the latest versions of the Prisma client ... and the errors rate drops from 10 per days to 10 during the last month.
So it's better 👍🏻 even if I can't say it's clearly linked to the Prisma client upgrade.

I think we can close this issue now. Many thanks!

Hate to bring this one back up, but I still see this on 3.14.0 frequently enough. I'm not sure exactly what triggers it (seems vaguely related to our deployments), but it persists until the affected node is recycled. I'm looking to move up to 4.x for other reasons, so hopefully I'll see it resolved there, but wanted to mention.

I am also seeing this issue with the latest prisma client

@ejames17 and @mattkindy could you open new issues with as much details as possible. What is the situation that causes this. If you could create a reproduction that would also help a lot.

I am also facing the same issue, prisma sometime connects to postgres DB correctly and someimes not.
I tested it on 2 providers( render, heroku) but I am still getting this problem.
It normally occurs while calling findMany function.
the error thrown is:
Can't reach database server at ...
Please make sure your database server is running at ...

I had a similar error and ran npm update prisma and that seems to have fixed it. Oddly nothing changed in my package.json, though that command reported that it changed 2 packages, so they were probably nested dependencies.

I'm facing the following error "sometimes", is there a way to automagically handle this error and reconnect?
PrismaClientKnownRequestError:
Invalid prismaClient_1.default.flight.findUnique() invocation in
/dist/database/flight.js:24:64
21 /
22 function getFlightInfo(flightId) {
23 return __awaiter(this, void 0, void 0, function () {
→ 24 const flightInfo = yield prismaClient_1.default.flight.findUnique(
Can't reach database server at ************.rds.amazonaws.com:3306
Please make sure your database server is running at .rds.amazonaws.com:3306.
at Zr.handleRequestError (/node_modules/@prisma/client/runtime/library.js:171:6414)
at Zr.handleAndLogRequestError (/node_modules/@prisma/client/runtime/library.js:171:5948)
at Zr.request (/node_modules/@prisma/client/runtime/library.js:171:5786)
at async t._request (/node_modules/@prisma/client/runtime/library.js:174:10455) {
code: 'P1001',
clientVersion: '4.11.0',
meta: {
database_host: '.rds.amazonaws.com',
database_port: 3306
}
}

I also get this error frequenly
Can't reach database server at ********..com:3306

@danimadmolil is your DB hosted with neon?

It seems that increasing the value of connect_timeout can solve this problem.
postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?schema=public&connect_timeout=30

@hclj37 legendary.

Because this have a high SEO, I would add that it is important to verify if you are dealing correctly with a lot of connections on the database connection pool. Make sure to use disconnect method await prisma.$disconnect(); after you finish your query or mutation.

Hi ,
I am getting this erro , could any one explain me why this is ossuring and what will be the solution for the same ?
Can't reach database server at <IP_ADDRESS>:3706
Please make sure your database server is running at <IP_ADDRESS>:3706.
frontend\node_modules\next\dist\compiled\next-server\app-route.runtime.dev.js:6:62609 {
code: 'P1001',
clientVersion: '5.8.1',
meta: {
modelName: 'locations',
database_host: '<IP_ADDRESS>',
database_port: 3706


[FEAT] add select method for choose Claude2 or Claude2.1

Is your feature request related to a problem? / 你想要的功能和什么问题相关？
I know I can change code for change v2 to v2.1 and build source code, but is possible in main branch put an option for select one of those models?
Describe the solution you'd like. / 你想要的解决方案是什么？
Form for choose or select one of 2 models
Describe alternatives you've considered. / 你考虑过的其他方案是什么？
No response
Additional context / 其他信息
No response

When using the claude-2 model, the response received is:
{error: {type: "permission_error", message: "Invalid model", code: "model_not_allowed"}} .
This indicates that usage of the claude-2 model is restricted by Claude.

I have pay account may be can test
How can test this kind of feature for select v2 or v2.1?
El mié, 29 nov 2023, 3:50 a. m., samanhappy @.***>
escribió:

When using the claude-2 model, the response received is:
{error: {type: "permission_error", message: "Invalid model", code:
"model_not_allowed"}} .
This indicates that usage of the claude-2 model is restricted by Claude.
—
Reply to this email directly, view it on GitHub
https://github.com/sunner/ChatALL/issues/639#issuecomment-1831466821,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADIWFAXKBKDPJ6PSOE34XLYG3ZOLAVCNFSM6AAAAAA72C72U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZRGQ3DMOBSGE
.
You are receiving this because you authored the thread.Message ID:
@.***>


Discord links are dead in README and website

Describe the bug
The Discord link https://discord.gg/KcVGzmCcWj, which is linked to from the README and the website, is dead :(

Seems the link has expired. Try this for now: https://discord.gg/EXjx7kb3


Inconsistency between website documentation and tutorials

Made the wording between the documentation (in semantics.xml) and tutorials regarding redelivering of messages consistent. Fixes #88.

@asbjornu no but we need to try to explain what "consumer dies" means: channel or connection closure, including TCP connection. This is a matter of wording, I'm not questioning the premise of #88.

@asbjornu if you don't feel confident about working on the wording, I can do that.

@asbjornu ok, re-worded and deployed.
You've been contributing increasingly useful changes. Can you please file a signed contributor agreement, as described in the README? It would greatly help us in accepting more contributions from you in the future :)
Thank you!

@michaelklishin Thanks. The new wording is indeed better. I'll sign the agreement as soon as I can get to it. :)


Remove: Peppy.Osu! version 2021.1211.0

## .\Tools\YamlCreate.ps1 -SkipPRCheck -Mode 5 -PackageIdentifier Peppy.Osu! -PackageVersion 2021.1211.0

[X] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Microsoft Reviewers: Open in CodeFlow

/azp run


Simplenote - Google Keep Killer

How about Simplenote? It's the only free cloud synced notes app that's available on ALL devices (Windows, Linux, Mac, Android & IOS) that I'm aware of. It's a real Google Keep killer. Joplin almost gets you there (for a price) but not quite. Thanks!

Looks cool adding it! Put on the list @ #374


somacore 1.0.26

Following our established procedure
Needed for https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/237
[sc-59686]

Thanks @gspowley !


feat(biome_css_analyze): implement noIrregularWhitespaceCss

Summary
This PR supercedes #3362 (there were some collisions between the css and the js rule and it was harder to fix than anticipated, so I started over) and implements the no-irregular-whitespace rule from stylelint.
My original implementation only checked for selectors, now this rule checks for complete rule blocks, regardless of type of rule (import rules, container rules, simple rule, you name it).
I was able to reuse most of the logic used in #3333 with some changes.
Test Plan
I tested some different @-rules, used the whitespace in the selector and in the rule itself. I also used all characters that are declared irregular.
closes #3264

@DerTimonius are you still interested in this PR?

@ematipico yes, I was wondering what is still missing

Just fix the conflicts and I'll merge it

@ematipico thanks, resolved the conflicts 🙂


Multicoin Integration - Wallet [ROUGH DRAFT]

User Story
As a multicoin wallet user, I want to be able to see all the coins I use in my wallet, and see the balance and transactions for each one separately.
It should be easy for me to send and receive funds for any coin the multicoin wallet supports manually. My node should be able to handle transactions in any coin the multicoin wallet supports without needing action from me.
As a user, I should not need to worry about adding or removing coins, my experience should be focused on interacting with the coins in my wallet that are active. My wallet should always be able to handle new transactions in coins it supports.
Functional Requirements
The multicoin wallet will always support all coins it can support. The user won't ever turn coins on or off. If they don't want to use a particular coin they can simply not create listings that accept that coin.

[ ] The wallet screen will need a way to select the current coin it is displaying.
[ ] The send and receive buttons should apply to the current coin being viewed.
[ ] There should be some way to see all your wallet balances at once, so you don't have to check them one by one.

This could be an "all coins" view, or a sidebar with just the coins and their balances, or the balance could be shown next to each coin name in the control to select a coin screen.

[ ] Add a coin parameter to the API calls detailed below.
QUESTION: should there be a Shapeshift feature to move funds from one of your coin wallets to another?

API Changes
The following calls will need a coin parameter sent to them.

GET wallet/address
GET wallet/balance
GET wallet/transactions
GET wallet/status
GET wallet/fees
POST wallet/spend
POST wallet/bumpfee
POST wallet/resyncblockchain

There should be some way to see all your wallet balances at once, so you don't have to check them one by one.

Can we please mirror existing multi-coin wallets on this one to make sure we are not taking on more scope than users are expecting.
For example, the Trezor web wallet does not do this.
I'll also defer to @morebrownies on how critical he thinks this would be.

QUESTION: should there be a Shapeshift feature to move funds from one of your coin wallets to another?

Would that need to be seperate from our existing shapeshift button. It would accomplish the same thing right?

Would it be better to have the list of preferred coins in the wallet?

Hmm... that seems like a settings piece of data to me.

Various sockets will need to update:

The wallet socket which comes for new transactions and when a transaction gets it's first confirmation will need the coin type added to it.
The walletUpdate socket which comes on a regular interval and gives us the current block
height which we can use to update the confirmations on a transaction should now return a list of data keyed by coin type so that we could have the current block height of each coin.

There should be some way to see all your wallet balances at once, so you don't have to check them one by one.

I'll also defer to @morebrownies on how critical he thinks this would be

Hey @jjeffryes @rmisio I think this would be nice to have, assuming the overhead isn't extremly complex. Giving a high level view across coins could surface information quicker to users, which is valuable, but I wouldn't say it's a show stopper if we couldn't get it working right away.

Would that need to be seperate from our existing shapeshift button? It would accomplish the same thing, right?

Yes and yes. The idea is it would be a streamlined "send 0.1 BTC to my BCH wallet" button. The same thing could be done by opening a wallet, copying the receive address, then switching to another wallet and pasting it into the send screen.
So not a requirement, just something I thought could be handy if users want to move coins around in their wallets. It could easily be deferred or made a stretch goal.

I see this issue covers high-level requirements. Is there any documentation/ideas/specifications on the low-level implementation of the web wallet? For example, how it will be implemented in the openbazaar-go server?

I see this issue covers high-level requirements. Is there any documentation/ideas/specifications on the low-level implementation of the web wallet? For example, how it will be implemented in the openbazaar-go server?

Not yet, we're still figuring that part out.


feature/skip preliminary analysis on dia

This PR adds the option to skip the preliminary steps of the dia analysis. (only do a single individual analysis and a single consensus analysis).
(please squash on merge ...)

Before reviewing the PR in details @jspaezp can you explain the impact of not doing preliminary step. I thought the idea with the preliminary step is to be able to generate the library for the final analysis @daichengxin ?

@ypriverol Absolutely!

The idea behind the feature is to allow two-stage runs of the pipeline, where a subset of files are used to generate the empirical library and then the extraction is done in all/the rest.
This is especially relevant because the empirical lib generation stage requires staging all the files (all .d + all .quant + fasta + predicted library) in the same compute environment/disk. (relevant discussion: https://twitter.com/J_my_sci/status/1744152837247095086)

btw... i dont believe any of the error in the ci/cd checks are caused by my changes ... I see a couple of files missing upstream and mamba not being able to generate environments.

We have to solve that, it is a work in progress because we have to move some files from the current server to PRIDE.

thanks @daichengxin for the review!

@jspaezp do you know which impact can have if you subselect a group of files compared to all the files in the final results?

I have not tested systematically this to be sure BUT. I would assume that (1) you could miss peptides that show up specifically in one of the conditions/files not used in the library construction. (2) You would have a slightly worse estimate of your FDR due to the smaller sample size.
I am not sure what public dataset could be used to test this hypothesis ... And I am assuming that data sets with more variability will be more prone to have changes depending on the analysis workflow. (I would be surprised if a 'cell line'+treatment dataset of 500 files looks any different if the library is done with the 500 files or with 100).

I have not tested systematically this to be sure BUT. I would assume that (1) you could miss peptides that show up specifically in one of the conditions/files not used in the library construction. (2) You would have a slightly worse estimate of your FDR due to the smaller sample size.
I am not sure what public dataset could be used to test this hypothesis ... And I am assuming that data sets with more variability will be more prone to have changes depending on the analysis workflow. (I would be surprised if a 'cell line'+treatment dataset of 500 files looks any different if the library is done with the 500 files or with 100).

Do you think in the logic we can use some of the SDRF information to do this selection?

I was thinking about this for a while and there might be a way, but it would certainly require a lot more nexflow plumbing that I really want to/can afford to devote right now ... In addition, I am not sure what cvparam could be used to denote that those should be used for the lib ... 1002752 ?? maybe ?
We could certainly have it as an open issue to implement the feature in the future (we could also discuss the right way to do it in the issue).
In other words, that is a much more complex feature than this PR attempts to be and I believe this feature by itself is complementary to that one.


access sqlite exception

i am trying access existing database from USB Pendrive in android 11, i'm getting below error.
SqliteException(4618): disk I/O error, disk I/O error (code 4618)
Causing statement: PRAGMA user_version;

It could be that sqlite3's locking is somehow not working on the USB stick for whatever reason, probably the device behaving weirdly. Are there other apps that might be reading the same database? Or are you opening the database on multiple threads / isolates maybe?
Also, this looks like a sqlite3 issue to me so I don't think there's much drift can do about it. I'm happy to help you with it, but I'll close this issue since it doesn't look drift-related.


fence.i privilege

Hi,
As the fence.i is the only instruction that maintain the instruction cache coherence. Fence.i will be implemented as an icache flush in a non-hardware coherent icache. So, what's the execution privilege level of the fence.i instruction? can user level  program execute fence.i?

Yes.
Note that in the Linux ABI, though, I$ flushing is done with a library call, not with fence.i.
https://github.com/bminor/glibc/blob/43b1048ab9418e902aac8c834a7a9a88c501620a/sysdeps/unix/sysv/linux/riscv/sys/cachectl.h#L27


Dockerfile in master branch differs from the layers found in the dockerhub latest tag

When looking at the DockerHub Digest for zcoinofficial/zcoind:latest, I see that the ENTRYPOINT differs from what the master branch's Dockerfile is showing...
Dockerhub Digest: ENTRYPOINT ["/bin/zcoind" "-printtoconsole"]
Dockerfile: ENTRYPOINT [ "/usr/bin/zcoind" ]
What is used to generate the tags in dockerhub?

Nice one for noticing this!
Yes, we did not use the one that is in the root of the repository for the current version. The reason is that the image will be so big (about 1 GB). So we decided to just put the release binary into the image instead. Here is its source:
FROM ubuntu:18.04

COPY zcoin-* /

RUN useradd -m -U zcoind
USER zcoind

WORKDIR /home/zcoind

ENTRYPOINT ["/bin/zcoind", "-printtoconsole"]

Thanks! I might want to eventually run my own image of zcoin for security reasons. This is helpful. Though I wonder what's using all of that space and whether it can be cleaned up or not.

The reason it takes a lot of space because our Dockerfile is not compact the image layers. Even though we remove the required packages in the last step but in reality, it does not remove those files from the previous layer. Even though we know how to fix it but currently we don't have enough resources to work on this.


Look into "cookies banner"

informational banner
opt in or out banner
text for this banner (likely info banner)

VGS to reach out to Sanna re: unicef's privacy policy copy
Create an info banner
Create a new page with Privacy Policy and Terms of Use

GDPR Compliance Notes: https://docs.google.com/document/d/1Zcm69aYwUATJua3kZvJCLskW4EAOP5Ant601FXSjvlc/edit

I argue that this can be closed since we switched Web Analytics to CloudFlare, and we don't use cookies anymore.

Sounds good!


Searching for CLI commands and flags (e.g. --schema or --watch) returns no results

Present here: https://www.prisma.io/docs/reference/tools-and-interfaces/prisma-cli/command-reference#overview

@jharrell the issue might be due to special characters not being picked up by algolia, such as -- in this case. Is this easy to resolve?

Closing this as this is outdated.


Added new URL on line 52

Updated URL to point to https://leeches.rosshendry.com/leeches

Thank you! I haven't touched this in years. I'm grateful to you for updating it.


[release-1.4] build: update wee8 to 7.9.317.14. (#287)

Signed-off-by: Piotr Sikora<EMAIL_ADDRESS>

This brings release-1.4 branch in sync with reality, since istio/proxy's release-1.4 branch points currently to this commit on the master branch.
Also, it looks that CODEOWNERS doesn't work here. cc @istio/release-managers-1-4


CollectionViewDemos Scrolling Pages don't scroll on Windows or Android

According to this issue, should use a Grid instead of StackLayout
https://github.com/dotnet/maui/issues/8990
https://github.com/dotnet/maui-samples/blob/fb630d735facd5929fab6b4f7410477b32f872ff/6.0/UserInterface/Views/CollectionViewDemos/CollectionViewDemos/Views/Scrolling/ScrollToByIndexPage.xaml

I replaced <StackLayout Margin="20"> with <Grid Margin="20"> and the vertical scroll started working.
This also worked:
<ScrollView>
    <VerticalStackLayout Margin="10">
        <CollectionView ItemsSource="{Binding Animals}"
        ...
        </CollectionView>
    </VerticalStackLayout>
</ScrollView>

Which way is proper/better?

Hi, I could reproduce the same behaviors with the latest revision (commit ac29ad78259df288d16e4126424bb52f86203dda) of the samples.
I'm testing on a android device (Xiaomi Mi Note 10 lite) and wrapping in Grids instead of StackLayouts seems to do the trick.
However, scrolling is very choppy and this is the behavior I'm also having in an another app(...)
I know @davidbritch worked on that part of the code some time ago, I bet it's a matter of priorities.
@iSeiryu, I found in many places that having a CollectionView within a ScrollView is generally not advised, because it can be confusing for the user (essentially, that's what you'll find on the web and stackoverflow).
There might also be issues with sizing as well, so I'll tend to the first approach : replacing every StackLayout and derivatives by Grids.
By the way, it seems to be the trend with dotnet MAUI to deprecate StackLayouts in favor of Grids instead (and they are a bit more complex to use)


Keyboard is hiding Progress bar when login button is pressed.

In login fragment when login button is pressed, Progress bar visible but keyboard is hiding the progress bar.

I want to work on this issue. :+1:


Prefer the form-specified wrapper over the global mapping when it is specified

Fixes #1258.
I'm not sure where in the tests this should go, guidance welcome.

Thank you for the pull request but we will stick with the current behavior. See https://github.com/plataformatec/simple_form/issues/1258#issuecomment-112382721


Improve fuzzer with better annotation processing and related issues

Description
Consider we need to fuzz the instance of the following class representing entity https://github.com/spring-projects/spring-petclinic/blob/main/src/main/java/org/springframework/samples/petclinic/owner/Owner.java
We need to save it to database in our tests later, so we would like to obtain at least one instance that follows all restrictions from annotations. It seems to be impossible with the current implementation of our fuzzer.
The restrictions have the following form:
@NotEmpty
@Digits(fraction = 0, integer = 10)
private String telephone;

Expected behaviour

We need to be able to process an instance that satisfies all annotated restrictions

It is suggested to allow to apply several mutations at once because trying to get a correct entity step by step may be too long.

We suggest to construct an entity that satisfies restrictions on the construction phase and start mutating it only after that.

It is also suggested to allow to create a fuzzed instance without any mutations - it may be useful to increase the coverage.

In case of spring-petclinic, including something like exception.stackTraceToString().length / 5 in fuzzer feedback seems to be enough to get entities that pass validation. The division by 5 is added to stop exceptions that only differ in some numbers from producing too many unique feedbacks.

Here's a test checking if fuzzing platform is able to generate values with no mutations, right now it fails.
@Test
fun `fuzzer can generate value without mutations`() {
    runBlocking {
            var seenEmpty = false
            withTimeout(1000) {
                runFuzzing(
                    { _, _ -> sequenceOf(Seed.Recursive<Unit, StringBuilder>(
                        construct = Routine.Create(emptyList()) { StringBuilder("") },
                        modify = sequenceOf(Routine.Call(emptyList()) { s, _ -> s.append("1") }),
                        empty = Routine.Empty { fail("Empty is called despite construct requiring no args") }
                    )) },
                    Description(listOf(Unit))
                ) { _, (s) ->
                    if (s.isEmpty()) {
                        seenEmpty = true
                        BaseFeedback(Unit, Control.STOP)
                    } else BaseFeedback(Unit, Control.CONTINUE)
                }
            }
            Assertions.assertTrue(seenEmpty) { "Unmodified empty string wasn't generated" }
    }
}

@IlyaMuravjov It's a bug


Alternate method

The article states that the only way to edit the custom support card is to go through the Windows Group Policy Management Editor. This is not the case.
The card can also be configured from Endpoint Manager.
The path is: Endpoint security>Antivirus>Create Policy>Windows Security Experience>Windows Defender Security Center. From there, the user can set the company name and add contact information to the support button.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e09345e7-0b81-b2f5-410f-54a855adc39d
Version Independent ID: c57c1b34-3999-54db-bf17-830af9dcf6bb
Content: Customize Windows Security contact information - Windows security
Content Source: windows/security/threat-protection/windows-defender-security-center/wdsc-customize-contact-information.md
Product: m365-security
Technology: windows-sec
GitHub Login: @Dansimp
Microsoft Alias: dansimp

Hi @colemank1, thank you for your feedback.
I'm adding the following information to the article:
"To enable the customized notifications and add the contact information in Intune, see Manage device security with endpoint security policies in Microsoft Intune and Settings for the Windows Security experience profile in Microsoft Intune."
And the sentence "This can only be done in Group Policy" is being moved to the step to enable the contact card.


Update error message to reference link command

Reference new(ish) railway link command in error message

Closing in favor of #99


Check for cygwin when adding .cmd to path

This is a tiny addon to @mattn's previous PR - it just includes cygwin (has('win32unix')) when adding .cmd to the executable path

I don't have Cygwin environment, bot maybe this is OK.


How to work with Docker containers using VS Code

This fixes #2147

Article checklist
ATTENTION: In your PR - add the ISSUE (topic form) # that this PR corresponds to. You can do this by also adding a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
If you have not already please go over our Resources Page for more tips and tricks on how to improve our overall technical writing.
Preliminary Checks - Formatting and Structure

[x] Does your article follow any of the suggested structure formats? - see example formats
[x] Is your article properly formatted in Markdown? - see Github Markdown guide
[x] Have you used the correct folder and file structure? - see Contribution Guidelines
[x] Is your article over 750 words? - Tool: Wordcounter.net
[x] Have you used our preferred heading structure? - H3 (###) and up
[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[x] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[x] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[x] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[x] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[x] Have you checked your article for readability? - Tool: Hemmingway
[ ] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines

@Njerikariuki Welcome to EngEd. 🎉🎉
Please add your author image as avatar.png in your authors folder.

Thank you @geoffrey45

I have aded the photo @geoffrey45

@Njerikariuki Very nice article. It is very clean for a first timer. I am hoping to see more work from you.

@hectorkambow This article is ready for final review.

plagiarism check done


Support for wildcard HTTP responses

As per the OpenAPI specification, it is allowed with wildcard http response codes (like 2xx, 3xx). Orval does not seem to support this.
https://learn.openapis.org/specification/paths.html#the-responses-object
What are the steps to reproduce this issue?
Given openapi-spec with wildcard response codes:
  /api/v1/users/me:
    get:
      tags:
      - user-resource
      summary: Fetch some metadata about the current logged in user
      operationId: getLoggedInUser
      responses:
        "2xx":
          description: Successful
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/UserDto'
        "4xx":
          description: Errors
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorDto'

What happens?
Orval seems to fail with
#/paths/~1api~1v1~1users~1me/get/responses must NOT have additional properties
What were you expecting to happen?
Not failing, but rather support the wildcard http response codes.
What versions are you using?
Package Version: 6.17.0

Its a bug in Swagger Validator you can try it here youself: https://apitools.dev/swagger-parser/online/
I opened a ticket there: https://github.com/APIDevTools/swagger-parser/issues/252


Follow Google style guide

Code style shall follow Google Java Style Guide as specified in Aion Code Conventions
Also add a CI test to confirm code is compliant

Reference https://github.com/google/google-java-format

Google code style removed from Aion Code Conventions here https://github.com/aionnetwork/aion/wiki/Aion-Code-Conventions/_compare/1c117765f1654d75346a01d649101c8e1ef7940d...9b6792a04387533970b282c8bb17d7f5838dd7da


cmd/gomobile: xcframework and catalyst support

Talked with @waylybaye offline, I decided to continue #63 and #45.
@waylybaye:

I changed the build process and it will create 3 frameworks for ios, simulator and catalyst. Every framework has a fat library contains both arm64 and x86_64 (except ios that only needs arm64).
Finally create one xcframework from all 3 fat frameworks and this xcframework contains every platform and architecture.

Review comments at https://go-review.googlesource.com/c/mobile/+/288752 should be mostly resolved.

@googlebot I consent.

@googlebot I consent

@dpwiese could you please add your consent too? Thanks!

@googlebot I consent


Fetching data from the localStorage on hard refresh via (vue-router) does not respect properties of given variables.

For example in vue-router:
router.beforeEach((to, from, next) => {
  let user = Vue.localStorage.get('user')
})

If the request comes from in-app routing user is indeed an object. But if we hard refresh the page user is still a string (JSON.stringified).
Update
After digging around I found out that the properties object was empty. So the type of the object user is a string, therefore no JSON object parsing has happened. The vue-router triggers the global beforeEach method before the vue-local-storage component could initialize its properties.

What if you declare your properties object on the Vue root. Will it change the behavior?

@pinguinjkeke declaring the properties in the Vue root does not change the behavior.
I dug a little deeper and found out that the $options object was empty in the beforeCreate method of the vue-localstorage mixin while in the beforeEach method of the vue-router. That's why the properties were empty.

A fix for this could be to initialize the properties where we register the plugin.
Vue.use(VueLocalstorage, {
  user: {
    type: Object,
    default: {}
  }
})

@anthdm the second parameter is for config. May be I should add third parameter to initialise it properly?

That would be nice.

@anthdm you could access you local storage in hooks.
router.beforeEach((to, from, next) => {
    if (to.matched.some(record => record.meta.requiresAuth)) {
        if(router.app.$localStorage.get('token')) {
            // do your stuff there

@ionmike  well, that's exactly the bug i'm describing.

Any ideas how to fix it? beforeEach is happended when component isn't being initialised, so bindings on beforeCreate inside package's mixin aren't setted up.

@pinguinjkeke 'm debugging it now and will try to come up with a fix.

@pinguinjkeke I found the cause of this bug.
The beforeEach from vue-router is triggered on its own mixin beforeCreate
So basically:
Vue.use(router)
Vue.use(localStorage) // if we use localStorage after router its broken

Vue.use(localStorage) // Works perfect
Vue.use(router)

The only thing we can do is make sure to use localStorage before vue-router.

After running this in production for a while, using localStorage before vue-router solved this issue.


Use transfer option once it is supported to improve the performance

https://github.com/denoland/deno/issues/8341

Deno 1.14 is released and it has [Zero copy feature](https://deno.com/blog/v1.14#zero-copy-arraybuffer-transfers-between-workers


9v Batteries are taped to the car

Batteries should probably be secured a little better. Both so that they can be changed easier and so that they look good.

Not relevant, 12v AA pack mounted to the bottom of the car.


vimPlugins.nvim-treesitter: fix update.py and update documentation

Description of changes
fixes https://github.com/NixOS/nixpkgs/issues/300471

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

cc @teto

You have a small typo in the commit message/PR title, vimPlug~s~ins

I think I’m incapable of typing the proper thing. They always become vimPlug sins.. maybe a sign?
Thanks for noticing ;)

Result of nixpkgs-review pr 321535 run on x86_64-linux 1

1 package built:

vimPluginsUpdater

Second commit could get squash merged in or rebased out


Rename string conversion functions to ToString()

closes https://github.com/Azure/azure-sdk-for-cpp/issues/1590
Pull Request Checklist
Please leverage this checklist as a reminder to address commonly occurring feedback when submitting a pull request to make sure your PR can be reviewed quickly:
See the detailed list in the contributing guide.

[x] C++ Guidelines
[x] Doxygen docs
[x] Unit tests
[x] No unwanted commits/changes
[x] Descriptive title/description

[x] PR is single purpose
[x] Related issue listed

[x] Comments in source
[x] No typos
[x] Update changelog
[x] Not work-in-progress
[x] External references or docs updated
[x] Self review of PR done
[x] Any breaking changes?

/azp run cpp - storage


Add abundance value to expression table

Can you please add the abundance to the expression table output? I'd like to have this function to make the boxplot of the individual abundances. Thanks!

That could be an option .. maybe for later release. In the meantime use the following example as a way to get what you are looking for:
tbl_expression <- hela_proteins %>%
    expression(knockdown/control) %>%
    export_analysis(knockdown/control, .analysis = "expression") %>%
    left_join(hela_proteins %>%  export_quant(),
              by = hela_proteins$identifier)

I added this functionality to v1.6.1, which will automatically output the abundance data in columns for each sample.


Make shared-fab.renderIcon and handlePointerDown protected

Change renderIcon and handlePointerDown methods of SharedFab from private back to protected to allow for extensibility

See https://github.com/material-components/material-web/issues/4200#issuecomment-1524003066


time_ago_in_words

A quick patch to support time_ago_in_words, with an options hash which will be passed to distance_of_time_in_words

Seems legit. Thanks! :)

Updated README to reflect these changes in @6084bfc.


Added an option to build a static linked version

Added an option to build a static linked version of diskspd to make it easier to build a single version and run it on various versions of Linux.  Based this new option on the make DEBUG=1 option to be consistent.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: hdornemann sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Hi @hdornemann, thank you very much - you'll need to sign this before I can merge your PR.

Sorry for the delay; i've  signed the CLA


Add a way to specify midi channel to message_on and message_off

Currently, Midi.message_on and Midi.message_off use by default the MIDI channel 1.
The implementation for message_on is currently as such:
let message_on ~note ~timestamp ?(volume = '\090') () =
  Event.create ~status:'\144' ~data1:note ~data2:volume ~timestamp

Where the status byte here is 144. (10100000 in binary)
The first bit of the status byte is always 1. The three next bit here are 010, which means the message is a Note On event. The last four bits are what interest us here.
They are set to 0000, which means that this message should be sent on MIDI channel 1.
There are a total of 16 MIDI channels available (so from 0000 to 1111.).
We want to add a named channel parameter to Midi.message_on and Midi.message_off in order to specify which channel to send the message to.
You can find here a reminder of how MIDI messages are encoded: https://cmtext.indiana.edu/MIDI/chapter3_channel_voice_messages.php

Hi @AryanGodara, I would say this issue is quite different from the volume issue, isn't it? This issue is about adding a parameter to message_on to specify the channel. The volume issue you're referring to is about enriching the Scale module and then using that enrichment to pass a different parameter to the volume parameter of message_on (i.e. pretty much what you've done already). Do you see what I mean with the difference?

Yes @pitag-ha , i see the difference between the two!
I'll focus on the vary volume task first and complete that asap!

Hey, since the vary volume issue is almost done (i'll fix the linting asap, i'm afk right now)
Can i continue with this issue?

Sure thing, go ahead! Thank you for contributing to this project. :-)

Sure thing, go ahead! Thank you for contributing to this project. :-)

It's a great experience for me also, learning so many new things 😁

To note: the bend_pitch function was just added to Midi.ml: you may want to also support ~channel as well for this message!

To note: the bend_pitch function was just added to Midi.ml: you may want to also support ~channel as well for this message!

Got it!
So i have to add channel support to

message_on
message_off
and pitch_bend

Is this correct?

yes!

For this my initial idea is to add an argument for channel :- [0,15] (16 channels)
Maybe we can use uintx of small size (0-15) if such small value is available in ocaml, to minimize error chances?
let message_on ~note ~timestamp ~volume ~channel () =

Then, do basic bit manipulation (simple addition actually)
10100000  (in decimal format) + whatever the input from channel is
(1 , 010 , 0000)
So, it will range from 10100000 to 10101111 in message on, and similar in message 2
[I'll go through the MIDI docs once again, and check if I'm setting the bits correctly)

Hello!
I think your idea works well,
For the type to use to represent the channel: I would not bother finding the perfect size for it.
I would just use an integer, and would raise an exception (Invalid_argument) if the user is trying to pass a value that is not legal!

Hello! I think your idea works well, For the type to use to represent the channel: I would not bother finding the perfect size for it.
I would just use an integer, and would raise an exception (Invalid_argument) if the user is trying to pass a value that is not legal!

I see, I'll have to look into raising exceptions. I haven't done it outside tutorials 😅

Also, isn't 144 -> 0b10010000 not 0b10100000
So, how is 010 the note_on event? Shouldn't it be 001 instead of 010


change unnecessary conditions

.blank? method checks for all falsy expressions including nil? empty?
that will make it more readable

An object is blank if it’s false, empty, or a whitespace string. For example, false, ”, ‘ ’, nil, [], and {} are all blank.
this is the documentation link :
https://apidock.com/rails/Object/blank%3F
so nil.blank? is true :
2.4.0 :001 > nil.blank?  => true
I guess it's readable even in normal English @michaelklishin


ODIS Logs and Metrics to Grafana

https://app.zenhub.com/workspaces/identity-backlog-board-60e61301c3dc55000e540b61/issues/celo-org/celo-monorepo/9100

Closing in favor of https://app.zenhub.com/workspaces/identity-backlog-board-60e61301c3dc55000e540b61/issues/celo-org/celo-monorepo/9100

@alecps - Closing this because work is complete


Delay before click release does't work properly

It seems to happen when i release my fingers realy quick or 2 fingers at the same time, the click releases instantly.

Can you upload your log file? Generate it in the Other Settings tab, you can also watch the time on your computer and remove the lines that are before/after the drag issue for me to clearly see only the related logs.

Can you upload your log file? Generate it in the Other Settings tab, you can also watch the time on your computer and remove the lines that are before/after the drag issue for me to clearly see only the related logs.

Logs_ThreeFingerDragOnWindows.txt
Thanks for replying! Here are related logs.

According to your logs, the release is due to the fact that the app detected that only one finger was present on the touchpad and provoked the cursor to move more than 10 pixels. Maybe I should put the move threshold as an option in the settings cause it can vary in function of screen DPI and touchpad hardware.

According to your logs, the release is due to the fact that the app detected that only one finger was present on the touchpad and provoked the cursor to move more than 10 pixels. Maybe I should put the move threshold as an option in the settings cause it can vary in function of screen DPI and touchpad hardware.

got it,sometimes it just releases randomly， thanks for replying!

But is never releases unless you releases your fingers right?

But is never releases unless you releases your fingers right?

oh of course, generally speaking it's perfect, jumpy movement also doesn't exist when i turn off the accelerate.

If you think that this is still a bug that should be fixed, you can keep the issue opened, otherwise I won't work on it.

If you think that this is still a bug that should be fixed, you can keep the issue opened, otherwise I won't work on it.

oh sorry, then I'll keep it opened

perfect solution by version <IP_ADDRESS>, thank you!!!


Not able to set enable_event_replication using artifactory_replication_config resource

Artifactory Enterprise Plus 7.33.12 rev
Terraform v0.13.7
After executing a terraform apply it seems the option enable_event_replication from the artifactory_replication_config resource does not apply.
Here's an example of a replication we had created using the replication resource
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  # artifactory_replication_config.npm-build-repo will be created
  + resource "artifactory_replication_config" "npm-build-repo" {
      + cron_exp                 = "0 00 05 * * ?"
      + enable_event_replication = true
      + id                       = (known after apply)
      + repo_key                 = "npm-build-repo"

      + replications {
          + enabled               = true
          + password              = (sensitive value)
          + socket_timeout_millis = (known after apply)
          + sync_deletes          = false
          + sync_properties       = true
          + sync_statistics       = true
          + url                   = (sensitive)
          + username              = (sensitive)
        }
    }

What we expect to see is when logging in the UI the 'Enable Event Replication' option should be set to true.

Hi @cw1797  ... I was not able to reproduce this issue. In my setup 'Enable Event Replication' was set ie: enabled when checked from UI.
Note: We have a Deprecation Message for 'artifactory_replication_config' resource. This resource has been deprecated in favor of the more explicitly name 'artifactory_push_replication' resource.

@cw1797 if you can reproduce this with the new resource, please reopen this or open a new ticket.


resolve data inconsistencies

Some PARMS.dat files and .VLD files have inconsistencies in the data that prevent them from loading properly in FC2. The inconsistencies are also present in FC1, but due to less error-checking on FC1, they get loaded into the database with the errors. We need to let Brenda know which ones so they can be fixed.

(we need to also let her know what the errors are, not just which ones have errors)

I am moving this into the current sprint because it is holding up Brenda's testing (she needs NTT to test and some of the data errors are in NTT).

I have found several inconsistencies in the freecen1 data (which we import into FC2). The inconsistencies are listed below.  Some of them are data errors with the files themselves (probably to be fixed by @FreecenBren or her coordinators), and some appear to be due to incorrect placement of files in the directory structure on the production server.  I am still a little fuzzy on the exact process of how the freecen1 files are put into specific directories on the server: whether they are checked into RCS and then checked out from RCS onto the servers, or whether they are simply copied into the locations on the filesystem.  For files that are in the wrong place, etc. should I just move them myself or does @lemon-ukgen or someone with FC1 do that?  (I'm hesitant to move FC1 files around without asking since I have not been involved with FC1 at all).
Here are the inconsistencies I found that should be resolved:
PARMS.DAT data errors: (these prevent the PARMS.DAT from loading in FC2, and also prevent any .VLD files for these year/county combinations from loading):

fixed/1851/DEVPARMS.DAT - blank place (district) names in pieces 1883 and 1885. In both cases, the line with a blank is also an 'a' row that shouldn't be there.
fixed/1861/DBYPARMS.DAT - blank place (district) name in piece 2512
fixed/1861/WILPARMS.DAT - blank place (district) name in piece 1309. Also the first 1309 row is a 'b' row before the 'a' row for 1309.
fixed/1871/LNDPARMS.DAT - blank place name in 512 (it is also an 'a' row, shouldn't be there).
fixed/1871/NTTPARMS.DAT - blank place names in 3493 (it is also marked as an 'n' row instead of 'a' or 'b'), 3494 (it is also marked as an 'a' row, it shouldn't be there). 3532 (it is marked as an 'a' row, shouldn't be there), 3542 (both 'a' and 'b' rows), 3543 (both 'a' and 'b' rows), 3547 (an unexpected 'a' row)
fixed/1891/GLAPARMS.DAT - the file is malformed beginning at 4940
fixed/1891/NTTPARMS.DAT - blank place names in 2659 ('b' row), 2660 ('a' row), also the file is malformed beginning at 2719

PARMS.DAT file inconsistencies:

there are two identical files (fixed/1861/DURPARMS.DAT and fixed/1861/durparms.dat). This doesn't cause an error because we ignore the lowercased filename file, but shows up as a warning in the processing log.
several files that are named ctyPARMT.DAT instead of ctyPARMS.DAT (T instead of S before the dot).  I'm guessing these are templates for the county parms files given to the coords before they add all the piece and place info? We ignore these, so not a major concern, just causes warning messages in the log. The ones there as of today are:
1871/WRYPARMT.DAT
1891/CAEPARMT.DAT
1891/CGNPARMT.DAT
1891/HEFPARMT.DAT
1891/HUNPARMT.DAT
1891/RADPARMT.DAT

.VLD filename/piece inconsistencies:

pieces/ABD/RS755168.VLD -- piece/par_num 168/55 not in parms. This appears to actually be ABD data, but this filename doesn't correspond to any piece/par_num specified in the parms.dat
pieces/ABD/RS756168.VLD -- piece/par_num 168/56 not in parms
pieces/ARL/HS512539.VLD -- piece/par_num 539/12 not in parms
pieces/HAM/RG094485.VLD -- piece/par_num not in parms
pieces/SOM/rg094497.vld -- ? doesn't seem to be an SOM piece, not sure where it should be instead of SOM

.VLD file location inconsistencies:

pieces/ABD/HS410899.VLD (piece not found in parms) -- I believe this is actually a WIG file and should be in the WIG directory?
pieces/ABD/HS510894.VLD (piece not found in parms) -- I believe this is actually a WIG file and should be in the WIG directory?
pieces/ABD/HS511319.VLD -- I believe should be in ANS, not ABD?
pieces/ABD/RS711137.VLD -- MOR? instead of ABD
pieces/BDF/RG121243.VLD -- HUN? instead of BDF
pieces/DUR/RG104248.VLD -- WRY? instead of DUR
pieces/ERY/RG093220.VLD -- WRY? instead of ERY
pieces/ERY/rg121413.vld -- ESS? instead of ERY
pieces/FLN/rg122870.vld -- piece/par_num not in parms
pieces/FLN/RG124611.VLD -- DEN? instead of FLN
pieces/FLN/RG124626.VLD -- DEN? instead of FLN
pieces/FLN/RG124627.VLD -- DEN? instead of FLN
pieces/LAN/RG103233.VLD -- LEI? instead of LAN
pieces/LAN/RG103293.VLD -- LEI? instead of LAN
pieces/LAN/RG103295.VLD -- LEI? instead of LAN
pieces/LAN/RG122511.VLD -- LEI? instead of LAN
pieces/OKI/hs410002.vld -- SHI instead of OKI
pieces/OKI/hs410005.vld -- SHI instead of OKI
pieces/OKI/hs410006.vld -- SHI instead of OKI
pieces/OKI/hs410008.vld -- SHI instead of OKI
pieces/OKI/hs410009.vld -- SHI instead of OKI
pieces/OKI/hs410010.vld -- SHI instead of OKI
pieces/OKI/hs410011.vld -- SHI instead of OKI
pieces/OKI/hs411001.vld -- SHI instead of OKI
pieces/OKI/hs411003.vld -- SHI instead of OKI
pieces/OKI/hs411004.vld -- SHI instead of OKI
pieces/OKI/hs411007.vld -- SHI instead of OKI
pieces/OKI/hs411012.vld -- SHI instead of OKI
pieces/OKI/hs412001.vld -- SHI instead of OKI
pieces/OKI/hs412003.vld -- SHI instead of OKI
pieces/OKI/hs412004.vld -- SHI instead of OKI
pieces/OKI/hs412007.vld -- SHI instead of OKI
pieces/OKI/hs412012.vld -- SHI instead of OKI
pieces/OKI/hs413007.vld -- SHI instead of OKI
pieces/OKI/hs413012.vld -- SHI instead of OKI
pieces/OKI/hs414012.vld -- SHI instead of OKI
pieces/SFK/RG121561.VLD -- NFK? instead of SFK
pieces/SSX/RG121437.VLD -- SFK? instead of SSX
pieces/STS/RG090767.VLD -- MDX? instead of STS
pieces/STS/RG092063.VLD -- WOR? instead of STS
pieces/STS/RG092064.VLD -- WOR? instead of STS
pieces/WRY/RG093588.VLD -- ERY? instead of WRY
pieces/WRY/RG093594.VLD -- ERY? instead of WRY
pieces/WRY/RG093595.VLD -- ERY? instead of WRY
pieces/WRY/RG093596.VLD -- ERY? instead of WRY
pieces/WRY/RG104788.VLD -- ERY? instead of WRY
pieces/WRY/RG104790.VLD -- ERY? instead of WRY
pieces/WRY/RG104798.VLD -- ERY? instead of WRY
pieces/WRY/RG104799.VLD -- ERY? instead of WRY
pieces/WRY/RG104800.VLD -- ERY? instead of WRY
pieces/NOR/RG121524.VLD -- NOR is not a valid chapman code, should be in pieces/NFK instead of pieces/NOR/
directory pieces/ANG -- ANG is not a valid chapman code
directory pieces/LDN -- LDN is not a valid chapman code
directory pieces/NOR -- NOR is not a valid chapman code
file pieces/PEE/PEEPARMS.DAT should not be in this directory.  NOTE: it is not identical to any of the fixed/18?1/PEEPARMS.DAT files, so we need to figure out which is the correct version of the file and make sure the correct version is in the proper fixed/18?1/ directory before deleting/moving this or replacing any in the fixed directory with this one.
unexpected .CSV file: pieces/RFW/HS414564.csv (there is also a .VLD for this piece in the directory)
unexpected .CSV file: pieces/RFW/HS516559.csv (there is also a .VLD)
unexpected .CSV file: pieces/RFW/HS518564.CSV (there is also a .VLD)
unexpected .CSV file: pieces/SOM/H0511922.csv (there is also a .VLD)

The PARMS files for the 1891 Census were all done at the start of FreeCEN and not by the COORDS who are here today. This is the Census year that FreeCEN frost started.
In fact many COORDS did not know how to do them and they were done for them.
I think the late Bob Muchamore and his team looked after that part.
When FreeCEN first started PIECES were often cross-over with neighbouring Counties.
The NTT/DBY 1891 is just one of these.
RG122719 was with the NTT Micro Fiche and was therefore was completed by that County.
When the DBY fiche came along that started with RG122720 as per the Fiche they were sent.
Many of them for that Census year were completed a few years back including the NTT 1891.
These match the PARMS for each County?
When the Census images were sent for the 1891 they were all on Micro Fiche.
The LDS would send a ‘County’ and the PARMS were done often from the Fiche.
The luxury we have to day to go onto the National archives Web Site example here http://discovery.nationalarchives.gov.uk/results/r?_q=RG+12%2F2659 was none existent.
So do we now change the PARMS to match the National Archives even though NTT 1891 is completed.
OK That is a little history.
What we need is what we are waiting for. The ability to be able to amend PARMS files online.
That is what Doug is working on. Not sure how far we have got with that?
Alternatively should I do it the old way and amend them manually and get them replaced online?
OK. I have picked a few others queries to do for now.

pieces/ABD/RS755168.VLD -- piece/par_num 168/55 not in parms.
This appears to actually be ABD data, but this filename doesn't correspond to any piece/par_num specified in the parms.dat
This one I do know the answer.
It needs being removed as it was named incorrectly as RS755168 and it has been replaced by RS756168 which had been wrongly names. See number 2 below.

pieces/ABD/RS756168.VLD -- piece/par_num 168/56 not in parms

pieces/ARL/HS512539.VLD -- piece/par_num 539/12 not in parms
This one is for Scotland.  It could well be named incorrectly as 539 is in the PARMS.
I will have to check with Rena on this.

SCT1851/539
Jura & Colonsay
Jura & Colonsay

pieces/HAM/RG094485.VLD -- piece/par_num not in parms
This needs removing from HAM as it is part of EWS and is online under EWS.
Why this is there I am not sure as it is not part of the HAM Parms for 1861.

pieces/SOM/rg094497.vld -- ? doesn't seem to be an SOM piece, not sure where it should be instead of SOM

This needs removing from SOM as it is part of EWS and is online under EWS.
Why this is there I am not sure as it is not part of the SOM Parms for 1861.
The DEN/FLN ones are of no surprise.
They have towns split between them and are forever changing counties.
These came on CD and are mixed together.

unexpected .CSV file: pieces/RFW/HS414564.csv (there is also a .VLD for this piece in the directory)
unexpected .CSV file: pieces/RFW/HS516559.csv (there is also a .VLD)
unexpected .CSV file: pieces/RFW/HS518564.CSV (there is also a .VLD)
unexpected .CSV file: pieces/SOM/H0511922.csv (there is also a .VLD)

These above need removing.   When I send Lemon VLD’s they are sent in 2 folders.

One for new VLD’s per county to be uploaded.

The other one for Amended VLD’s
This folder means that there is already a VLD been uploaded previously.
This need removing first and then the Amended one needs to be added.
This looks like there might be some historic ones lurking about before Lemon took on the job of doing FreeCEN uploads. It looks like some CSV’s were sent and also manged to get uploaded.
This needs to be addressed so that only VLD’s are uploaded and anything other than them should be rejected.

Sorry it is the granddaughters 18th birthday and we are now off to the party.
If you let me have a list of the ones that you need me to have a look at and do something with please send me an email and I will see what I can find.
Brenda
From: dougkdev<EMAIL_ADDRESS>Sent: Sunday, October 30, 2016 1:26 AM
To: FreeUKGen/FreeCENMigration<EMAIL_ADDRESS>Cc: Brenda<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [FreeUKGen/FreeCENMigration] resolve data inconsistencies (#154)
I have found several inconsistencies in the freecen1 data (which we import into FC2). The inconsistencies are listed below. Some of them are data errors with the files themselves (probably to be fixed by @FreecenBren https://github.com/FreecenBren  or her coordinators), and some appear to be due to incorrect placement of files in the directory structure on the production server. I am still a little fuzzy on the exact process of how the freecen1 files are put into specific directories on the server: whether they are checked into RCS and then checked out from RCS onto the servers, or whether they are simply copied into the locations on the filesystem. For files that are in the wrong place, etc. should I just move them myself or does @lemon-ukgen https://github.com/lemon-ukgen  or someone with FC1 do that? (I'm hesitant to move FC1 files around without asking since I have not been involved with FC1 at all).
Here are the inconsistencies I found that should be resolved:
PARMS.DAT data errors: (these prevent the PARMS.DAT from loading in FC2, and also prevent any .VLD files for these year/county combinations from loading):

fixed/1851/DEVPARMS.DAT - blank place (district) names in pieces 1883 and 1885. In both cases, the line with a blank is also an 'a' row that shouldn't be there.
fixed/1861/DBYPARMS.DAT - blank place (district) name in piece 2512
fixed/1861/WILPARMS.DAT - blank place (district) name in piece 1309. Also the first 1309 row is a 'b' row before the 'a' row for 1309.
fixed/1871/LNDPARMS.DAT - blank place name in 512 (it is also an 'a' row, shouldn't be there).
fixed/1871/NTTPARMS.DAT - blank place names in 3493 (it is also marked as an 'n' row instead of 'a' or 'b'), 3494 (it is also marked as an 'a' row, it shouldn't be there). 3532 (it is marked as an 'a' row, shouldn't be there), 3542 (both 'a' and 'b' rows), 3543 (both 'a' and 'b' rows), 3547 (an unexpected 'a' row)
fixed/1891/GLAPARMS.DAT - the file is malformed beginning at 4940
fixed/1891/NTTPARMS.DAT - blank place names in 2659 ('b' row), 2660 ('a' row), also the file is malformed beginning at 2719

PARMS.DAT file inconsistencies:

there are two identical files (fixed/1861/DURPARMS.DAT and fixed/1861/durparms.dat). This doesn't cause an error because we ignore the lowercased filename file, but shows up as a warning in the processing log.
several files that are named ctyPARMT.DAT instead of ctyPARMS.DAT (T instead of S before the dot). I'm guessing these are templates for the county parms files given to the coords before they add all the piece and place info? We ignore these, so not a major concern, just causes warning messages in the log. The ones there as of today are:
1871/WRYPARMT.DAT
1891/CAEPARMT.DAT
1891/CGNPARMT.DAT
1891/HEFPARMT.DAT
1891/HUNPARMT.DAT
1891/RADPARMT.DAT

.VLD filename/piece inconsistencies:

pieces/ABD/RS755168.VLD -- piece/par_num 168/55 not in parms. This appears to actually be ABD data, but this filename doesn't correspond to any piece/par_num specified in the parms.dat
pieces/ABD/RS756168.VLD -- piece/par_num 168/56 not in parms
pieces/ARL/HS512539.VLD -- piece/par_num 539/12 not in parms
pieces/HAM/RG094485.VLD -- piece/par_num not in parms
pieces/SOM/rg094497.vld -- ? doesn't seem to be an SOM piece, not sure where it should be instead of SOM

.VLD file location inconsistencies:

pieces/ABD/HS410899.VLD (piece not found in parms) -- I believe this is actually a WIG file and should be in the WIG directory?
pieces/ABD/HS510894.VLD (piece not found in parms) -- I believe this is actually a WIG file and should be in the WIG directory?
pieces/ABD/HS511319.VLD -- I believe should be in ANS, not ABD?
pieces/ABD/RS711137.VLD -- MOR? instead of ABD
pieces/BDF/RG121243.VLD -- HUN? instead of BDF
pieces/DUR/RG104248.VLD -- WRY? instead of DUR
pieces/ERY/RG093220.VLD -- WRY? instead of ERY
pieces/ERY/rg121413.vld -- ESS? instead of ERY
pieces/FLN/rg122870.vld -- piece/par_num not in parms
pieces/FLN/RG124611.VLD -- DEN? instead of FLN
pieces/FLN/RG124626.VLD -- DEN? instead of FLN
pieces/FLN/RG124627.VLD -- DEN? instead of FLN
pieces/LAN/RG103233.VLD -- LEI? instead of LAN
pieces/LAN/RG103293.VLD -- LEI? instead of LAN
pieces/LAN/RG103295.VLD -- LEI? instead of LAN
pieces/LAN/RG122511.VLD -- LEI? instead of LAN
pieces/OKI/hs410002.vld -- SHI instead of OKI
pieces/OKI/hs410005.vld -- SHI instead of OKI
pieces/OKI/hs410006.vld -- SHI instead of OKI
pieces/OKI/hs410008.vld -- SHI instead of OKI
pieces/OKI/hs410009.vld -- SHI instead of OKI
pieces/OKI/hs410010.vld -- SHI instead of OKI
pieces/OKI/hs410011.vld -- SHI instead of OKI
pieces/OKI/hs411001.vld -- SHI instead of OKI
pieces/OKI/hs411003.vld -- SHI instead of OKI
pieces/OKI/hs411004.vld -- SHI instead of OKI
pieces/OKI/hs411007.vld -- SHI instead of OKI
pieces/OKI/hs411012.vld -- SHI instead of OKI
pieces/OKI/hs412001.vld -- SHI instead of OKI
pieces/OKI/hs412003.vld -- SHI instead of OKI
pieces/OKI/hs412004.vld -- SHI instead of OKI
pieces/OKI/hs412007.vld -- SHI instead of OKI
pieces/OKI/hs412012.vld -- SHI instead of OKI
pieces/OKI/hs413007.vld -- SHI instead of OKI
pieces/OKI/hs413012.vld -- SHI instead of OKI
pieces/OKI/hs414012.vld -- SHI instead of OKI
pieces/SFK/RG121561.VLD -- NFK? instead of SFK
pieces/SSX/RG121437.VLD -- SFK? instead of SSX
pieces/STS/RG090767.VLD -- MDX? instead of STS
pieces/STS/RG092063.VLD -- WOR? instead of STS
pieces/STS/RG092064.VLD -- WOR? instead of STS
pieces/WRY/RG093588.VLD -- ERY? instead of WRY
pieces/WRY/RG093594.VLD -- ERY? instead of WRY
pieces/WRY/RG093595.VLD -- ERY? instead of WRY
pieces/WRY/RG093596.VLD -- ERY? instead of WRY
pieces/WRY/RG104788.VLD -- ERY? instead of WRY
pieces/WRY/RG104790.VLD -- ERY? instead of WRY
pieces/WRY/RG104798.VLD -- ERY? instead of WRY
pieces/WRY/RG104799.VLD -- ERY? instead of WRY
pieces/WRY/RG104800.VLD -- ERY? instead of WRY
pieces/NOR/RG121524.VLD -- NOR is not a valid chapman code, should be in pieces/NFK instead of pieces/NOR/
directory pieces/ANG -- ANG is not a valid chapman code
directory pieces/LDN -- LDN is not a valid chapman code
directory pieces/NOR -- NOR is not a valid chapman code
file pieces/PEE/PEEPARMS.DAT should not be in this directory. NOTE: it is not identical to any of the fixed/18?1/PEEPARMS.DAT files, so we need to figure out which is the correct version of the file and make sure the correct version is in the proper fixed/18?1/ directory before deleting/moving this or replacing any in the fixed directory with this one.
unexpected .CSV file: pieces/RFW/HS414564.csv (there is also a .VLD for this piece in the directory)
unexpected .CSV file: pieces/RFW/HS516559.csv (there is also a .VLD)
unexpected .CSV file: pieces/RFW/HS518564.CSV (there is also a .VLD)
unexpected .CSV file: pieces/SOM/H0511922.csv (there is also a .VLD)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/FreeUKGen/FreeCENMigration/issues/154#issuecomment-257124046 , or mute the thread https://github.com/notifications/unsubscribe-auth/ANC91hU2MdxBUFxhuJJsN6O3wEakegdKks5q4-QTgaJpZM4KbCK0 .

Hi Brenda,
Thanks for this.  I think we should work closely with TNA on this - e.g.
routing people to their pages for the explanation, much as we do to GENUKI
from FreeREG for C of E Parishes.
We need to get the TNA to say 'high quality transcriptions may be
available, for free, from FreeCEN' on all their pages with a link to us,
ideally above the link to Ancestry.
I've started a shopping list to take to TNA....
Best wishes,
Pat
- -
Dr Pat Reynolds
Executive Director
Free UK Genealogy  http://www.freeukgenealogy.org.uk/
A charity registered in England and Wales, number 1096940, transitioning to
a Charitable Incorporated Organisation registered in England and Wales,
number 1167484
VAT registration: 233 0105 70
+44 1904 541411 +44 7943 145387
36 Albemarle Road, York, YO23 1ER, UK
On 30 October 2016 at 09:53, Brenda<EMAIL_ADDRESS>wrote:

The PARMS files for the 1891 Census were all done at the start of FreeCEN
and not by the COORDS who are here today. This is the Census year that
FreeCEN frost started.
In fact many COORDS did not know how to do them and they were done for
them.
I think the late Bob Muchamore and his team looked after that part.
When FreeCEN first started PIECES were often cross-over with neighbouring
Counties.
The NTT/DBY 1891 is just one of these.
RG122719 was with the NTT Micro Fiche and was therefore was completed by
that County.
When the DBY fiche came along that started with RG122720 as per the Fiche
they were sent.
Many of them for that Census year were completed a few years back
including the NTT 1891.
These match the PARMS for each County?
When the Census images were sent for the 1891 they were all on Micro Fiche.
The LDS would send a ‘County’ and the PARMS were done often from the Fiche.
The luxury we have to day to go onto the National archives Web Site
example here http://discovery.nationalarchives.gov.uk/
results/r?_q=RG+12%2F2659 was none existent.
So do we now change the PARMS to match the National Archives even though
NTT 1891 is completed.
OK That is a little history.
What we need is what we are waiting for. The ability to be able to amend
PARMS files online.
That is what Doug is working on. Not sure how far we have got with that?
Alternatively should I do it the old way and amend them manually and get
them replaced online?
OK. I have picked a few others queries to do for now.

pieces/ABD/RS755168.VLD -- piece/par_num 168/55 not in parms.
This appears to actually be ABD data, but this filename doesn't correspond
to any piece/par_num specified in the parms.dat
This one I do know the answer.
It needs being removed as it was named incorrectly as RS755168 and it has
been replaced by RS756168 which had been wrongly names. See number 2 below.

pieces/ABD/RS756168.VLD -- piece/par_num 168/56 not in parms

pieces/ARL/HS512539.VLD -- piece/par_num 539/12 not in parms
This one is for Scotland. It could well be named incorrectly as 539 is in
the PARMS.
I will have to check with Rena on this.

SCT1851/539
Jura & Colonsay
Jura & Colonsay

pieces/HAM/RG094485.VLD -- piece/par_num not in parms
This needs removing from HAM as it is part of EWS and is online under EWS..
Why this is there I am not sure as it is not part of the HAM Parms for

pieces/SOM/rg094497.vld -- ? doesn't seem to be an SOM piece, not sure
where it should be instead of SOM

This needs removing from SOM as it is part of EWS and is online under EWS..
Why this is there I am not sure as it is not part of the SOM Parms for
1861.
The DEN/FLN ones are of no surprise.
They have towns split between them and are forever changing counties.
These came on CD and are mixed together.

unexpected .CSV file: pieces/RFW/HS414564.csv (there is also a .VLD
for this piece in the directory)
unexpected .CSV file: pieces/RFW/HS516559.csv (there is also a .VLD)
unexpected .CSV file: pieces/RFW/HS518564.CSV (there is also a .VLD)
unexpected .CSV file: pieces/SOM/H0511922.csv (there is also a .VLD)

These above need removing. When I send Lemon VLD’s they are sent in 2
folders.

One for new VLD’s per county to be uploaded.

The other one for Amended VLD’s
This folder means that there is already a VLD been uploaded previously.
This need removing first and then the Amended one needs to be added.
This looks like there might be some historic ones lurking about before
Lemon took on the job of doing FreeCEN uploads. It looks like some CSV’s
were sent and also manged to get uploaded.
This needs to be addressed so that only VLD’s are uploaded and anything
other than them should be rejected.

Sorry it is the granddaughters 18th birthday and we are now off to the
party.
If you let me have a list of the ones that you need me to have a look at
and do something with please send me an email and I will see what I can
find.
Brenda
From: dougkdev<EMAIL_ADDRESS>Sent: Sunday, October 30, 2016 1:26 AM
To: FreeUKGen/FreeCENMigration<EMAIL_ADDRESS>Cc: Brenda<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [FreeUKGen/FreeCENMigration] resolve data inconsistencies
(#154)
I have found several inconsistencies in the freecen1 data (which we import
into FC2). The inconsistencies are listed below. Some of them are data
errors with the files themselves (probably to be fixed by @FreecenBren <
https://github.com/FreecenBren> or her coordinators), and some appear to
be due to incorrect placement of files in the directory structure on the
production server. I am still a little fuzzy on the exact process of how
the freecen1 files are put into specific directories on the server: whether
they are checked into RCS and then checked out from RCS onto the servers,
or whether they are simply copied into the locations on the filesystem. For
files that are in the wrong place, etc. should I just move them myself or
does @lemon-ukgen https://github.com/lemon-ukgen or someone with FC1 do
that? (I'm hesitant to move FC1 files around without asking since I have
not been involved with FC1 at all).
Here are the inconsistencies I found that should be resolved:
PARMS.DAT data errors: (these prevent the PARMS.DAT from loading in FC2,
and also prevent any .VLD files for these year/county combinations from
loading):

fixed/1851/DEVPARMS.DAT - blank place (district) names in pieces 1883
and 1885. In both cases, the line with a blank is also an 'a' row that
shouldn't be there.
fixed/1861/DBYPARMS.DAT - blank place (district) name in piece 2512
fixed/1861/WILPARMS.DAT - blank place (district) name in piece 1309.
Also the first 1309 row is a 'b' row before the 'a' row for 1309.
fixed/1871/LNDPARMS.DAT - blank place name in 512 (it is also an 'a'
row, shouldn't be there).
fixed/1871/NTTPARMS.DAT - blank place names in 3493 (it is also marked
as an 'n' row instead of 'a' or 'b'), 3494 (it is also marked as an 'a'
row, it shouldn't be there). 3532 (it is marked as an 'a' row, shouldn't be
there), 3542 (both 'a' and 'b' rows), 3543 (both 'a' and 'b' rows), 3547
(an unexpected 'a' row)
fixed/1891/GLAPARMS.DAT - the file is malformed beginning at 4940
fixed/1891/NTTPARMS.DAT - blank place names in 2659 ('b' row), 2660
('a' row), also the file is malformed beginning at 2719

PARMS.DAT file inconsistencies:

there are two identical files (fixed/1861/DURPARMS.DAT and
fixed/1861/durparms.dat). This doesn't cause an error because we ignore the
lowercased filename file, but shows up as a warning in the processing log..
several files that are named ctyPARMT.DAT instead of ctyPARMS.DAT (T
instead of S before the dot). I'm guessing these are templates for the
county parms files given to the coords before they add all the piece and
place info? We ignore these, so not a major concern, just causes warning
messages in the log. The ones there as of today are:
1871/WRYPARMT.DAT
1891/CAEPARMT.DAT
1891/CGNPARMT.DAT
1891/HEFPARMT.DAT
1891/HUNPARMT.DAT
1891/RADPARMT.DAT

.VLD filename/piece inconsistencies:

pieces/ABD/RS755168.VLD -- piece/par_num 168/55 not in parms. This
appears to actually be ABD data, but this filename doesn't correspond to
any piece/par_num specified in the parms.dat
pieces/ABD/RS756168.VLD -- piece/par_num 168/56 not in parms
pieces/ARL/HS512539.VLD -- piece/par_num 539/12 not in parms
pieces/HAM/RG094485.VLD -- piece/par_num not in parms
pieces/SOM/rg094497.vld -- ? doesn't seem to be an SOM piece, not sure
where it should be instead of SOM

.VLD file location inconsistencies:

pieces/ABD/HS410899.VLD (piece not found in parms) -- I believe this is
actually a WIG file and should be in the WIG directory?
pieces/ABD/HS510894.VLD (piece not found in parms) -- I believe this is
actually a WIG file and should be in the WIG directory?
pieces/ABD/HS511319.VLD -- I believe should be in ANS, not ABD?
pieces/ABD/RS711137.VLD -- MOR? instead of ABD
pieces/BDF/RG121243.VLD -- HUN? instead of BDF
pieces/DUR/RG104248.VLD -- WRY? instead of DUR
pieces/ERY/RG093220.VLD -- WRY? instead of ERY
pieces/ERY/rg121413.vld -- ESS? instead of ERY
pieces/FLN/rg122870.vld -- piece/par_num not in parms
pieces/FLN/RG124611.VLD -- DEN? instead of FLN
pieces/FLN/RG124626.VLD -- DEN? instead of FLN
pieces/FLN/RG124627.VLD -- DEN? instead of FLN
pieces/LAN/RG103233.VLD -- LEI? instead of LAN
pieces/LAN/RG103293.VLD -- LEI? instead of LAN
pieces/LAN/RG103295.VLD -- LEI? instead of LAN
pieces/LAN/RG122511.VLD -- LEI? instead of LAN
pieces/OKI/hs410002.vld -- SHI instead of OKI
pieces/OKI/hs410005.vld -- SHI instead of OKI
pieces/OKI/hs410006.vld -- SHI instead of OKI
pieces/OKI/hs410008.vld -- SHI instead of OKI
pieces/OKI/hs410009.vld -- SHI instead of OKI
pieces/OKI/hs410010.vld -- SHI instead of OKI
pieces/OKI/hs410011.vld -- SHI instead of OKI
pieces/OKI/hs411001.vld -- SHI instead of OKI
pieces/OKI/hs411003.vld -- SHI instead of OKI
pieces/OKI/hs411004.vld -- SHI instead of OKI
pieces/OKI/hs411007.vld -- SHI instead of OKI
pieces/OKI/hs411012.vld -- SHI instead of OKI
pieces/OKI/hs412001.vld -- SHI instead of OKI
pieces/OKI/hs412003.vld -- SHI instead of OKI
pieces/OKI/hs412004.vld -- SHI instead of OKI
pieces/OKI/hs412007.vld -- SHI instead of OKI
pieces/OKI/hs412012.vld -- SHI instead of OKI
pieces/OKI/hs413007.vld -- SHI instead of OKI
pieces/OKI/hs413012.vld -- SHI instead of OKI
pieces/OKI/hs414012.vld -- SHI instead of OKI
pieces/SFK/RG121561.VLD -- NFK? instead of SFK
pieces/SSX/RG121437.VLD -- SFK? instead of SSX
pieces/STS/RG090767.VLD -- MDX? instead of STS
pieces/STS/RG092063.VLD -- WOR? instead of STS
pieces/STS/RG092064.VLD -- WOR? instead of STS
pieces/WRY/RG093588.VLD -- ERY? instead of WRY
pieces/WRY/RG093594.VLD -- ERY? instead of WRY
pieces/WRY/RG093595.VLD -- ERY? instead of WRY
pieces/WRY/RG093596.VLD -- ERY? instead of WRY
pieces/WRY/RG104788.VLD -- ERY? instead of WRY
pieces/WRY/RG104790.VLD -- ERY? instead of WRY
pieces/WRY/RG104798.VLD -- ERY? instead of WRY
pieces/WRY/RG104799.VLD -- ERY? instead of WRY
pieces/WRY/RG104800.VLD -- ERY? instead of WRY
pieces/NOR/RG121524.VLD -- NOR is not a valid chapman code, should be
in pieces/NFK instead of pieces/NOR/
directory pieces/ANG -- ANG is not a valid chapman code
directory pieces/LDN -- LDN is not a valid chapman code
directory pieces/NOR -- NOR is not a valid chapman code
file pieces/PEE/PEEPARMS.DAT should not be in this directory. NOTE: it
is not identical to any of the fixed/18?1/PEEPARMS.DAT files, so we need to
figure out which is the correct version of the file and make sure the
correct version is in the proper fixed/18?1/ directory before
deleting/moving this or replacing any in the fixed directory with this one.
unexpected .CSV file: pieces/RFW/HS414564.csv (there is also a .VLD
for this piece in the directory)
unexpected .CSV file: pieces/RFW/HS516559.csv (there is also a .VLD)
unexpected .CSV file: pieces/RFW/HS518564.CSV (there is also a .VLD)
unexpected .CSV file: pieces/SOM/H0511922.csv (there is also a .VLD)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <
https://github.com/FreeUKGen/FreeCENMigration/issues/154#
issuecomment-257124046> , or mute the thread <https://github.com/
notifications/unsubscribe-auth/ANC91hU2MdxBUFxhuJJsN6O3wEakeg
dKks5q4-QTgaJpZM4KbCK0> .
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/FreeUKGen/FreeCENMigration/issues/154#issuecomment-257141877,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ANXTHTK_wh0WxdEOcxQ8n0B7-aQlMtaSks5q5GkwgaJpZM4KbCK0
.

dependent on #99

Answering some of @dougkdev's question about the FreeCEN1 files, as much a know:
The PARMS files
( eg ~apache/hosts/freecen/fixed/1891/AGYPARMS.DAT)
These live in CVS at UKGEN-FreeCEN/fixed/.... I have occasionally been asked to update one but very infrequently.
The VLD files
( eg /raid/freecen/pieces/ABD/HS410169.VLD )
I've never been clear what mechanism was used to update these before I got involved but Brenda and I came with a tactical fix because updates had stalled for a while. This involves Brenda updating a Dropbox folder with new or updated VLD files organised in their XYZ subdirectories  and a script to work through these putting them into the file tree on the master server.
The script is at ~apache/hosts/freecen/bin/update-pieces.rb and its use outlined in the UKGEN-FreeCEN/README) file in CVS.
FreeCEN1 build process
After updating the VLD files on the master I would run the database update process, also in the README. AIUI that combines the PARMS deployed in the last handoff into ~apache/hosts/freecen) with the VLD in /raid/freecen/pieces to build the database.
So that's the process I know of to date. I reckon the VLD files can be fixed in place without my involvement. For consistency I think any changes to PARMS files should be committed back to UKGEN-FreeCEN CVS.
I am happy to help where I can but there's nothing here that I own in particular :)

Thanks, @lemon-ukgen.

I have resolved most of the inconsistencies.  I made corrections to the 7 PARMS.DAT files that had errors in them and emailed the corrected versions to @FreecenBren for review. The corrected versions are loaded on FC2 (test2), but not checked in to freecen1 CVS until approved by her. I have also relocated the approximately 60 .VLD files to the correct county directories so they load on FC2.
Three .VLD file errors remain:

ARL/HS512539.VLD - this piece is from Scotland and @FreecenBren  is going to check with Rena about it. (possibly named incorrectly?)

ABD/RS756168.VLD - This one does not load due to mismatch between file name and parms.dat information. (note: it is also listed as a file that didn't load on FC1 at http://www.freecen.org.uk/errors.html)

NTT/RG122683.VLD - this file has a row at the end with "C    001886000" instead of valid data.

@FreecenBren  has reviewed the 7 amended PARMS.DAT files and they have been checked in to the CVS repository to be picked up with the next freecen1 update.

It is my opinion that we can go ahead and close this story.  The large number of inconsistencies that were the reason I opened the story have been resolved (the 7 parms.dat files that wouldn't load on FC2 due to errors and the 60+ VLD files that wouldn't load on FC2 because they were in the incorrect county directories). There are a handful of questions still being looked into, but we are now at a small number of inconsistencies, about the number that might arise after the FC1 update for any given month.


Upgrade and improve support for small-scale variants

Regarding bbefd239bab87396746ec9fd190a69501e540f81 I fail to reproduce the issue now. The cluster gets up and running just fine, on k3s.

On one GKE cluster I ran into https://github.com/containerd/containerd/issues/3604#issuecomment-526373394, and with the workaround the metrics exporter crashlooped. Everything works on a GKE cluster with dockerd instead of containerd nodes. I'll trust that containerd gets upgraded on next node upgrade.


The header of the csv file generated by --csv-raw option is not correct

The header should print the name of the benchmark as well.
iters,time,cycles,cpuTime,utime,stime,maxrss,minflt,majflt,nvcsw,nivcsw,allocated,numGcs,bytesCopied,mutatorWallSeconds,mutatorCpuSeconds,gcWallSeconds,gcCpuSeconds
elimination/toNull/streamly,1,1.6136415e-2,35418132,1.6138e-2,0.0,0.0,3706880,0,0,0,3,95989920,93,3304,1.5653e-2,1.5653e-2,2.62e-4,3.24074e-4

This issue obstructs automated tools from correctly parsing the file based on the field names.


Blue Mountains/Nogrod

Build by teagan75, Ghorgen and Jerle17

If we can get a dwarvish stable installed, I can add a fast travel point here if it will be done in time for release.

Sorry I will tell my brother...completely forgot about this :)

Going to call this build complete


use a favicon in the docs ?

I was thinking of using the doc as a small demo using the sphinx-contrib logohttps://avatars.githubusercontent.com/u/31936682?s=200&v=4.
I have a small favicon generator to get all the format at once
Bonus: I would like to change the theme for pydata-sphinx-theme and explain how to use nox in the contribution section

Using that icon is a great idea! I didn't really have an idea for what logo to use, so this would be perfect 😀
You can absolutely also switch to the pydata-sphinx-theme!


Remove remaining instances of networking not using NSURLSession

We need to ensure there is no networking remaining that is not using NSURLSession (see  this).
A cursory search found two instances of NSURLConnection:
https://github.com/cloudant/CDTDatastore/blob/c407be82aa0d821628a9b59658e9f32e38344426/CDTDatastore/touchdb/TDMultipartUploader.m#L54
https://github.com/cloudant/CDTDatastore/blob/c407be82aa0d821628a9b59658e9f32e38344426/CDTDatastore/touchdb/TDRemoteRequest.m#L249

@willholley also points out:

https://github.com/cloudant/CDTDatastore/blob/c407be82aa0d821628a9b59658e9f32e38344426/CDTDatastore/touchdb/TDReachability.h#L12 and
https://github.com/cloudant/CDTDatastore/blob/92dfcd1e6eaa52109200658400f8ea7958091f2a/CDTDatastoreTests/CDTReplicationTests.m#L79 might cause problems

that list was from https://developer.apple.com/videos/play/wwdc2015/719/ where they state (9 mins in) that the following will break under IPv6:
uint32_t
in_addr
sockaddr_in
inet_aton
gethostbyname
gethostbyname2(hostname,AF_INET)

Closing as this library is now EOL and this issue won't be fixed here.
The issue has been tagged EOL in case any forks want to import it.


Can we please make a new release

The version has been updated already in the setup.py, we just need to change the date in updates, and push that to pypi. Can we please do that?
@jpoullet2000

Done


Drop kubelet-https removed in 1.22

Connections to kubelets have used HTTPS unconditionally since 1.19
Needed to land https://github.com/openshift/kubernetes/pull/862
/assign @sttts

/lgtm


tippecanoe-decode

Hello, thanks for your incredible tool !
Since I've updated my tippecanoe with brew upgrade tippecanoe and get 1.27.6,
if I do a tippecanoe-decode data.mbtiles z x y > example.geojson, I get a non JSON object, commas are missing between layers objects.
I havent this issue with tippecanoe 1.17.1.
thanks for yours replies

Please reopen if you can give details of the problem.

OK, I do have a way to reproduce this after all.
➤ ./tippecanoe -f -o foo.mbtiles tl_2010_06001_tract10.json tl_2015_06001_roads.json
For layer 0, using name "tl_2010_06001_tract10"
For layer 1, using name "tl_2015_06001_roads"
28066 features, 2865801 bytes of geometry, 5490 bytes of separate metadata, 621779 bytes of string pool
  99.9%  14/2640/6347
➤ ./tippecanoe-decode foo.mbtiles 11 328 791 |  json_pp -f json -t json > /dev/null
, or ] expected while parsing array, at character offset 171154 (before " "type": "Feature", ...") at /usr/bin/json_pp5.18 line 45.

Seems OK when decoding either an entire mbtiles file or a single PBF, but not a single tile with multiple layers from an mbtiles file.

Thanks for the report. This will be fixed in https://github.com/mapbox/tippecanoe/pull/541

Fix is merged


Use timers internally

Fixes #23

Yeah I can see Timers being kinda tricky with negative speeds. Only thing I can think of is to reverse the frame order rather then the actual speed and keep the speed >0, though this is overly complicated and also introduces more state.