Function to update/add object in realm data models

I want to auto add a timestamp field for each realm data model object . It will be helpful for the synchorization (merging decisions) between the devices and server. Thus I need a function to add new realm data model object base on the realm data model's keys.
Currently, from what i understand is that one can only add a new object within the migration block.  So my current proposed solution is that i will do a reflection on the realm data models to get the keys, then create a migration block and add new objects for each key. After which, i will update the database. Which is definitely not the right approach!
Is it possible to have a function that allows me to create if not exist update for the realm data model field?
realm.write {
 realm.add(<realm data model>,"<realm data model object field name>", update: true)
}

I am using realm swift for development.
Thanks
Perwyl

@perwyl as it turns out, we already have such a method! The following code snippet should do the trick, using Realm.create(_:value:update:).
 realm.write {
  realm.create(<realm data model>,["propertyName" : newValue], update: true)
}

@segiddins  i got this error msg *** Terminating app due to uncaught exception 'RLMException', reason: 'Missing property value' *** First throw call stack: libc++abi.dylib: terminating with uncaught exception of type NSException
What i did was,

add the realmObject to realm
get a reflected object on realmObject
create a timestamp for each key in realmObject
update realmObject with the timestamps

PS: the timestamp keys does not exist in realmObject
`var realmObject = RLMObject()
realm.write{
realm.add(realmObject, update: true)
}
var reflectedObject = reflect(realmObject)
var timestampList = String: NSDate
for var index = 0; index < reflectedObject ; ++index {
var property : String  = reflectedObject[index].0 + "TimeStamp"
timeStampList[property] = NSDate()
}
realm.write {
realm.create(RLMObject.self, value:timeStampList, update: true)
}`

It looks as though you're mixing Realm Objective-C and Realm Swift (passing RLMObject() to realm.add(_:)), which is unsupported. I've just added a note at the very top of our online documentation to reflect this.
Also, from our Object docs (link):

In Realm you define your model classes by subclassing Object and adding properties to be persisted. You then instantiate and use your custom subclasses instead of using the Object class directly.

So instead of creating Object() directly, you should define your model as a subclass (like class MyModel: Object { /* properties here... */}) and create instances of that directly. You can read more about modeling data in Realm in the Models section of our Swift documentation.
Finally, you should be setting properties directly rather than constructing a property name at runtime from the reflection, which I'm confused as to why you would want to do this at the moment.
So here's another take on your code from my assumptions in what you're trying to accomplish:
import RealmSwift

// MARK: Model definitions

// schema v0
// class Person: Object {
//   dynamic var name = ""
// }

// schema v1
class Person: Object {
  dynamic var name = ""
  dynamic var nameTimeStamp = NSDate()
}

// MARK: Migration

setSchemaVersion(1, Realm.defaultPath, { migration, oldSchemaVersion in
  if oldSchemaVersion < 1 {
    // Nothing to do, since the default value of `Person().nameTimeStamp` is already the current date.
  }
})

Thanks @jpsim  for the quick reply. Sorry for my bad naming, RLMObject  is just a class name representation. I want to create the properties at runtime is that for each class, I had a lot of properties.
Just thinking it might be good for me to create (dynamically ) new properties (timestamps) at run time (i need a timestamp for each property in a class). The reason for the timestamps is that it will be helpful for synchorization (merging decisions) between the devices and server.
class RLMObject: Object {
 dynamic var name = ""
...
}

I see! If you want a dynamic list of timestamps, I'd still do this in a type-safe manner defined at compile-time like so:
class TimeStamp: Object {
  dynamic var propertyName = ""
  dynamic var lastModified = NSDate()
}
class BaseModel: Object {
  let propertyTimeStamps = List<TimeStamp>()
}
class Person: BaseModel {
  dynamic var name = ""
}

This does mean that every time you edit the property on an object, you'll have to update its respective entry in propertyTimeStamps, but this will give you the behavior you want, without requiring you to duplicate every property on your models.

@jpsim thanks ! Let me try it out.
And may i know the purpose of this method ?
 realm.write {
  realm.create(<realm data model>,["propertyName" : newValue], update: true)
}

You can see the API documentation for that method here:

Create an Object with the given value.

Creates or updates an instance of this object and adds it to the Realm populating the object with the given value.

When ‘update’ is ‘true’, the object must have a primary key. If no objects exist in the Realm instance with the same primary key value, the object is inserted. Otherwise, the existing object is updated with any changed values.


i'd like to publish this data in RPZ format

hello. i am paul vixie, and along with vernon schryver i co-created the DNS firewall system known as RPZ. see https://dnsrpz.info/ for more information about that, but briefly, it's a way to publish DNS policy information in a way that many different RDNS servers can subscribe in real time. the original implementation was in BIND9 but it is now supported in PowerDNS as well.
i realize that the "hosts" corpus doesn't change all that often, and that the real time update capability of DNS RPZ will probably not come into play very often here. however, there is a dearth of freely available DNS RPZ content -- most publishers are commercial. with this community's permission, i would set up a cron job to "git pull" from this repo, convert the results into DNS RPZ format, and if there's any change from the prior content, run an "rndc reload" to cause the new content to be absorbed and pushed out.
i'm doing this now with the suspect-networks.io feed, which it now occurs to me, you may want to incorporate into your corpus here. in any case i won't take this action without consensus, and steven black told me by e-mail that opening an issue here is the way to measure such consensus.
one final detail -- RPZ is free, unencumbered by patent, and i am not proposing to charge money for the DNS RPZ subscription version of this corpus. i will also give full source-credit whenever i discuss it. i don't ask for money or glory, only the opportunity to extend the reach of your work and the utility of DNS RPZ.
other information can be found here: https://en.wikipedia.org/wiki/Paul_Vixie

As long as all the licensing terms of use are observed, I don't have any problem with it. I'm sure other members may want to put in a bit more due diligence before commenting either way, but I'm fine with anything that makes the Internet a safer place for more people.

It has my thumbs up. Especially if Paul is part of it ;).

Hi Paul @vixie, is this something this repo might do natively?  Is this the best spec to study??
We also have Issue $47 and #139 which call for listing hosts in ip6 as format as well, which is among our next steps, I think.  How do you deal with this?
If you'd like to use the hosts files here and crank them into another format, I have no objection to that, bearing in mind that comments from origin sources should be preserved insofar as the licensing requires it.

i don't think the repo can do it natively, since it involves speaking the DNS protocols, specifically NOTIFY, IXFR, and AXFR. yes, that version of the spec is fine. we handle ipv6 addresses just fine. the way i'd handle it is by parsing your "hosts" files and generating qname and nsdname triggers, having nxdomain actions. i can preserve comments as non-operative TXT records in the rpz results, but there is no annotation in text of an actual rpz "hit", so those TXT records would only show up during diagnostic work.

Well, to reference https://github.com/StevenBlack/hosts/issues/139 again, Steven's latest comments about starting a server instance and API for hosts file distributions may come into play again here if he wants to go the extra distance to host a minimal DNS server to distribute it that way, as well. If it's all in the same environment using shared resources and the DNS is configured to only distribute the policies and not act as a fully functional DNS, I don't think it's that much of a leap at all.
The counter to the above, of course, would be I'm sure Paul would rather prefer something he could have full control over. Seeing as how the specs are only a few months old, I think he is looking to get a working proof of concept off the ground to solidify his model in general before having a bunch of other people jumping in. Of course I can also see how he might prefer to delegate out some of the work, as well, depending on what his personal schedule, etc., is like.
Either way I think this is up to Paul and his needs. And then up to Steven, as well, if Paul decides to go the route of partnering up to some degree. In any case, I think either decision remains beneficial to the consumers at large.

@vixie understood.  Go for it!

@ScriptTiger i have no need of control, but i note that i care more about this than most others, so am willing to do the work. also the specs are five years old but the rfc draft is new. RPZ has been out in the world since 2011 or so. what's missing is solid well-curated free content for it. that's where "hosts" comes in.
@StevenBlack ok will do.

I think in your personal position there may be some political things to consider, as well, on behalf of your project. Personally taking on this project would put your project at odds with Google on the analytics and ads pieces, as well as all the other companies who provide such legitimate services that this particular community simple does not endorse.  If Steven were to perform the service and you to simply point people in that direction as an available content host, but not officially endorse the content and present it more as an option or example, it might give you a bit more plausible deniability. I am not sure if this really applies to your particular case as I am not so intimately involved with your project, but it may be something to consider depending on the level of visibility you wish your project to have.

Also, just to clarify some terminology. I have been doing a bit of reading on this, how does the content here fit in with the "reputation data" model? We do indeed have many third parties contribute to this list, but we don't score domains or anything like that, we simply put them on the list or don't. Is this a different model from the earlier specs involving reputation data? Or have they both always coexisted as just different features of the same thing to rely on reputation data and/or supplied domain lists? After some reading I was kind of left with the impression the reputation data model was rather core to the project, transferring over from your earlier work with the Mail Abuse Prevention System.

Hey @ScriptTiger please go easy on visitors.  Paul @vixie, who needs no introduction, simply came here for permission and clearance, and it's all-good.

Lol, I totally understand. I say these things out of personal interest, so I do hope it's not misunderstood.

@ScriptTiger RPZ is not limited to reputation, it even has some non-security purposes such as configuration management (I use it for localhost handling, for example.) There is no scoring.

@StevenBlack Firstly, I want to thank you and others who have contributed and keep doing so for maintaining the lists. I've personally deployed numerous pi-hole's for family and friends and without this work, the options would have been non-existent.
Re-opening this issue since I don't think the the conversion of the lists to DNS RPZ format happened. With the community's permission,  would  it be alright if I volunteer to do it ( giving full credit to this repo/source)? @vixie

Hi Swapneel @swapneelp!  This strikes me as a very good idea, and you have my blessing and support to take this as far as you'd like to take it.
Over in Issue #739, Add support for Little Snitch Rule Group Subscriptions format we are considering offering domains in the Little Snitch format, too.  I think this product would live alongside each hosts file variant, in the same folders.
So you can carry this forward as its own separate project/repo,  or integrate RPZ into the products of this repo, I'm happy either way.

@StevenBlack Thank you for the quick response and support.
My suggestion is to create a separate repo to begin with and get it up and running. The scripts & documentation can then be included in this repo.  I will then offer a DNS RPZ feed via my recursive DNS server which other users running DNS RPZ can do a DNS zone transfer(AXFR/IXFR). This way, users purely interested in using the feed will not have to do the conversion from the current format to the DNS RPZ format. They will merely have to point their recursive DNS server to my master server. If more people volunteer & offer zone transfers, the better it is for the community.
For people who would not want to do a zone transfer, I will also include the scripts and document the steps for doing the conversion.  @vixie

@swapneelp, I understand what DNS RPZ is, but I'm not too keen on the "DNS RPZ format." I know after your last post you're planning on doing a lot of work in relation to this, but can you just quickly post a quick sample of what the format is, like a sample of the file contents, etc.?

@ScriptTiger Sure. DNS RPZ uses zone files. Below is a sample output from abuse.ch(The site appears to be facing some issues right now, I've shared the archived link of the page from wayback machine along with the original source,
Wayback machine - urlhaus.abuse.ch DNS RPZ page
Original Source - urlhaus.abuse.ch DNS RPZ feed

Ah, okay. I'm familiar with regular BIND zone files, but when searching about how to format DNS RPZ I kept getting the bracketed and blocked formats, but I think those must have been for the RPZ policies. So just to clarify, the format that you're looking to convert to is as show below?
some.domain.com CNAME . ; Comment comment comment
another.domain.com CNAME . ; Comment comment comment

If this is true, this format could also be used for BIND servers without RPZ, as well, they just won't get distributed via RPZ, etc.

@ScriptTiger Yes.
The sample link I had shared as well as the code output in your last comment results in the NXDOMAIN RPZ action. Just like any firewall, DNS RPZ is basically made up of TRIGGERS & ACTIONS. If anyone wants to learn more about the various TRIGGERS & ACTIONS, please see draft-vixie-dnsop-dns-rpz-00
For the latter part of your comment, that's not the only value. As mentioned earlier in this comment, with DNS RPZ, one can use various TRIGGERS. For example if I want to serve NXDOMAIN for the StevenBlack/hosts for the IP addresses <IP_ADDRESS>, <IP_ADDRESS> (Refer "Client IP Address" Trigger) but exempt <IP_ADDRESS> from that policy action (Refer "PASSTHRU" Action). Can we do this policy based(TRIGGERS & ACTIONS) natively without DNS RPZ ? No.
The distribution of the zone data can happen using dns zone transfers(AXFR/IXFR) and is not a feature of the DNS RPZ.

see also https://dnsrpz.info/ which i linked in the head of this thread. and note that the draft is up to -04 now; -00 was just the initial rough cut. see https://tools.ietf.org/html/draft-vixie-dns-rpz-04 for newest.

I completely understand the triggers and actions and the value of RPZ, don't get me wrong at all there, it's definitely awesome. I'm just thinking about who else can use the same format. As long as the RPZ-specific options aren't used, such as rpz-passthru., rpz-drop., and rpz-tcp-only., and the file headers are removed, it seems to me there are other DNS and firewall applications, for instance, that could use the same format, as well.
For example, the following may be what most are familiar with when it comes to a normal BIND CNAME record declaration.
example.com.               IN      CNAME   host1.example.com.
However, the IN class is actually defaulted to and can be omitted entirely to make it look like the same format as we want for DNS RPZ.
example.com.               CNAME   host1.example.com.
And there are other applications, as I said, which use the same format, as well. So I'm not at all dissing RPZ, I am just further clarifying the format for those that may want to borrow your product for other uses and make the most out of it.

Besides the header bits (i.e. $TTL 30, @ SOA rpz.urlhaus.abuse.ch. hostmaster.urlhaus.abuse.ch.<PHONE_NUMBER> 300 1800 604800 30, NS localhost.), do these RPZ files look about right? Would they function for the purposes of DNS RPZ?
https://scripttiger.github.io/alts/

@ScriptTiger  yes, it works
(or at least bind doesn't complain & the 2 names I tried were blocked :)
It would be nice if you could summarize things tho - eg: one line instead of 1384:
$ grep '\.2o7\.net' db.stevenblack-rpz | wc -l
1384

could be one line
*.2o7.net  CNAME  .

@ler762, check out https://github.com/StevenBlack/hosts/issues/530.

@ScriptTiger   I've looked at #530 but .. what specifically?
I'm already using privoxy & have a script that converts the blocklist to a privoxy action file and comments out redundant lines - eg:
.privacyassistant.net
# blocked by prev: www.privacyassistant.net
# blocked by prev: ext.privacyassistant.net
# blocked by prev: www.ext.privacyassistant.net

If you're talking about

If contributors can agree on a standard way to document RegEx

I like your idea of
#*.angiemktg.com
for saying all of angiemktg.com is to be blocked

noting, DNS RPZ is meant to be the standard format of DNS policy filtering. i'd hate to hear that we'd failed. .foo works, but foo. does not. that is, wildcards work in RPZ the way they work in DNS, not the way they work in the shell. we did not support regex because we have multimillion-rule RPZ's and we have to be able to search them in constant time. regex and any other string search is quadratic.
also noting, the file is a zone, and zones can be transferred, and zone transfers can be triggered by a NOTIFY (reducing update latency), secured with TSIG (ensuring authenticity and possibly offering access control as well), and incrementally transferred using IXFR, meaning only the deltas have to be transmitted. we can update a thousand subscribers of a multi-million rule RPZ every second -- and we do. i realize that rsync and other tools could do this for files. but with DNS RPZ it's all done by DNS.
what i do with policy rulesets that are not in RPZ format is fetch them in a cron job, diff each against prior, and if there is a difference, i copy it into place and tell BIND9 to reload that zone. BIND9 has an option called ixfr-from-differences whereby it loads the new zone, computes the deltas from the prior zone, and then offers that delta as IXFR to each subscriber, who is told with NOTIFY, and who may know a TSIG preshared key. the only latency is in the cron job frequency; all else is real time.
see also this work:
https://www.researchgate.net/publication/228895300_Modern_DNS_as_a_coherent_dynamic_universal_database
a full text of this article is online at:
http://family.redbarn.org/~vixie/dnsind.pdf

@ler762, check the RRPZ files:
https://scripttiger.github.io/alts/

On Saturday, 24 August 2019 06:35:44 UTC ScriptTiger wrote:

@ler762, check the RRPZ files:
https://scripttiger.github.io/alts/

ty! RRPZ looks like it won't be nec'y and may overblock (there could be
subdomains of the shared domain which are not malicious).
how frequently are these files updated? i don't want to fetch more often than
that.
--
Paul

RRPZ looks like it won't be nec'y and may overblock

We disagree.  As far as I'm concerned, there's no such thing as overblocking things like 2o7.net
but if somebody wants to allow certain names they can do
appleglobal.112.2o7.net CNAME rpz-passthru.  ; unbreak apple.com

@ler762, check the RRPZ files:

sweet!  Again, bind doesn't complain but I get weird results - like appleglobal.112.2o7.net isn't blocked!   I was thinking it was me not understanding/misconfiguring something, but I'm not so sure now..  It'd be nice if others would test
Starting out with
$ grep '\.2o7\.net' db.stevenblack-rrpz
*.2o7.net CNAME .
and.co.uk.102.122.2o7.net CNAME .
;<IP_ADDRESS> appleglobal.112.2o7.net #breaks apple.com
;<IP_ADDRESS> applestoreus.112.2o7.net #breaks apple.com
ehg-moma.hitbox.com.112.2o7.net CNAME .
expedia.ca.112.2o7.net CNAME .
infomart.ca.112.2o7.net CNAME .
infospace.com.112.2o7.net CNAME .
metro.co.uk.102.122.2o7.net CNAME .
overstock.com.112.2o7.net CNAME .
sa.aol.com.122.2o7.net CNAME .
; <IP_ADDRESS> appleglobal.112.2o7.net #[affects Apple site]
flyingmag.com.122.2o7.net CNAME .
homepjlconline.com.112.2o7.net CNAME .
nysun.com.112.2o7.net CNAME .
popsci.com.122.2o7.net CNAME .
; <IP_ADDRESS> survey.112.2o7.net #[affect Sprint.com]
bcbsks.com.102.112.2o7.net CNAME .
www.ikea.122.2o7.net CNAME .

appleglobal.112.2o7.net isn't blocked
If I comment out all the extra 2o7.net names
$ grep '2o7\.net' db.stevenblack-rrpz
; 2o7.net -- server side tracking
2o7.net CNAME .
*.2o7.net CNAME .
; and.co.uk.102.122.2o7.net CNAME .
; appleglobal.112.2o7.net       CNAME .
;<IP_ADDRESS> appleglobal.112.2o7.net #breaks apple.com
;<IP_ADDRESS> applestoreus.112.2o7.net #breaks apple.com
; ehg-moma.hitbox.com.112.2o7.net CNAME .
; expedia.ca.112.2o7.net CNAME .
; infomart.ca.112.2o7.net CNAME .
; infospace.com.112.2o7.net CNAME .
; metro.co.uk.102.122.2o7.net CNAME .
; overstock.com.112.2o7.net CNAME .
; sa.aol.com.122.2o7.net CNAME .
; <IP_ADDRESS> appleglobal.112.2o7.net #[affects Apple site]
; flyingmag.com.122.2o7.net CNAME .
; homepjlconline.com.112.2o7.net CNAME .
; nysun.com.112.2o7.net CNAME .
; popsci.com.122.2o7.net CNAME .
; <IP_ADDRESS> survey.112.2o7.net #[affect Sprint.com]
; --WTF-- bcbsks.com.102.112.2o7.net CNAME .
; www.ikea.122.2o7.net CNAME .

appleglobal.112.2o7.net. is blocked
Uncomment just the one line
bcbsks.com.102.112.2o7.net CNAME .

and looking up appleglobal.112.2o7.net. works :(
If it makes a difference, I'm testing with bind
Version: 9.11.5-P4-5~bpo9+1-Debian

ty! RRPZ looks like it won't be nec'y and may overblock (there could be
subdomains of the shared domain which are not malicious).

As referenced in https://github.com/StevenBlack/hosts/issues/530, technically it would be impossible to wild card with 100% accuracy because the needed information just is not present in the hosts file format. However, I will reference the RRPZ description from the download page before continuing:

RRPZ
Similar to the RPZ format except that multiple child domains of a single blocked parent domain are reduced to a single wild card child domain of the shared common domain. Please note if multiple child domains of a common parent domain are present but the parent domain itself is not, those entries will not be reduced as the parent itself is not blocked and may have other child domains that shouldn't be blocked, as well.

That description should directly address your concerns. Basically in order to generate a list with wild cards certain assumptions are made, such as if a common shared parent domain itself is blocked and has child domains that are blocked as well, then the entire domain and all child domains should be blocked. This is an "if and only if" scenario, so it's a bit of a compromise or hybrid. If there are no child domains of any given domain blocked, then child domains for that given domain will not be blocked. And, as stated in the description, if multiple domains share a common parent domain, all such child domains of said parent domain will only be blocked if and only if that parent domain is explicitly blocked, as well.
However, I am, of course, all ears for how this algorithm might be improved if there are any suggestions to that effect.

how frequently are these files updated? i don't want to fetch more often than
that.

I regenerate all of the data for my entire website shortly after every new release from @StevenBlack. Obviously a generation script is used, but the script itself is triggered manually by myself and overseen for quality control purposes. Feel free to check out the website's repo here:
https://github.com/ScriptTiger/scripttiger.github.io
And the website's commit log:
https://github.com/ScriptTiger/scripttiger.github.io/commits/master
You can also find a similar chronicling of events posted as they happen from my Twitter:
https://twitter.com/scripttiger

sweet! Again, bind doesn't complain but I get weird results - like appleglobal.112.2o7.net isn't blocked! I was thinking it was me not understanding/misconfiguring something, but I'm not so sure now.. It'd be nice if others would test

@vixie could probably answer that much better than I. Whether it's a legitimate bug or if there's some context we are not taking into account, I can't think of a better person to consult with than the imagineer himself.

@ScriptTiger what is a child domain?  I think you want to use, subdomain.  That's the universally used nomenclature.

I'll make a separate post for this as it's just me thinking and not an actual solution or expert opinion of any kind.
The example @ler762 uses is appleglobal.112.2o7.net becomes unblocked once bcbsks.com.102.112.2o7.net is blocked, despite having *.2o7.net in place. This would lead me to believe that all parent domains of bcbsks.com.102.112.2o7.net are graced with an override to ignore the *.2o7.net rule.

ty! RRPZ looks like it won't be nec'y and may overblock (there could be
subdomains of the shared domain which are not malicious).

This leads me to believe that, again, this is part of that compromise to ensure domains are not over-blocked, and perhaps evidence to that effect, as implemented and carried out within RPZ itself. Explicitly blocked domains will, of course, remain blocked. However, explicitly blocked child, grandchild, etc., domains of an explicitly blocked parent domain with a wild card get an override when their child, grandchild, etc., domains are present in any further explicitly blocked domains.
For example, after *.2o7.net is implemented, let's assume all child, grandchild, etc., domains become successfully blocked. However, once bcbsks.com.102.112.2o7.net is also explicitly blocked, com.102.112.2o7.net, 102.112.2o7.net, and 112.2o7.net are overriden with this new rule and ignore  *.2o7.net.
This might actually be seen as an improvement upon my originally intended compromise. Consider the following comments:
;<IP_ADDRESS> appleglobal.112.2o7.net #breaks apple.com
;<IP_ADDRESS> applestoreus.112.2o7.net #breaks apple.com

Because of the way RPZ works, these domains are actually not blocked because of bcbsks.com.102.112.2o7.net providing an override for the 112.2o7.net parent domain, saving apple.com from being broken. All you would have to do is uncomment those lines, as stated, to explicitly block them if you don't care that apple.com gets broken.
Thoughts?

@ScriptTiger what is a child domain? I think you want to use, subdomain. That's the universally used nomenclature.

@StevenBlack, my apologies!!! I would have corrected that in my last comment had I read yours first, but apparently we were both literally typing our comments at the same time and you beat me to it and now my wording looks silly and like I am specifically trying to be annoying in the face of your previous comment, which I am not. Further references will be made to subdomains and not child domains, noted!

ScriptTiger wrote on 2019-08-24 17:09:

ty! RRPZ looks like it won't be nec'y and may overblock (there could be
subdomains of the shared domain which are not malicious).

As referenced in #530 https://github.com/StevenBlack/hosts/issues/530,
technically it would be impossible to wild card with 100% accuracy ...
RRPZ
Similar to the RPZ format except that multiple child domains of a
single blocked parent domain are reduced to a single wild card child
domain of the shared common domain. Please note if multiple child
domains of a common parent domain are present but the parent domain
itself is not, those entries will not be reduced as the parent
itself is not blocked and may have other child domains that
shouldn't be blocked, as well.

That description should directly address your concerns. ...
However, I am, of course, all ears for how this algorithm might be
improved if there are any suggestions to that effect.

when i generate an rpz i usually put in two rules for any known-bad domain:
example.com CNAME .
*.example.com CNAME .
this follows from my belief that the identifier was granted based on
public trust, which has not been met in this case, and it ought to be
revoked, but there's no mechanism for that, so we're doing this.
so if i knew of three bad names which shared a common parent, such as:
foo.example.com
bar.example.com
baz.example.com
i would generate the following RPZ content:
foo.example.com CNAME .
*.foo.example.com CNAME .
bar.example.com CNAME .
*.bar.example.com CNAME .
baz.example.com CNAME .
*.baz.example.com CNAME .
in no case would i connote irresponsibility to the parent, since it
might be an effective TLD and i would never want to block, e.g., *.com
or *.co.uk or *.co.nz or similar.

how frequently are these files updated? i don't want to fetch more
often than
that.

...
You can also find a similar chronicling of events posted as they happen
from my Twitter:
https://twitter.com/scripttiger

ty!

sweet! Again, bind doesn't complain but I get weird results - like
appleglobal.112.2o7.net isn't blocked! I was thinking it was me not
understanding/misconfiguring something, but I'm not so sure now..
It'd be nice if others would test

@vixie https://github.com/vixie could probably answer that much better
than I. Whether it's a legitimate bug or if there's some context we are
not taking into account, I can't think of a better person to consult
with than the imagineer himself.
in the instructions for setting up an RPZ, i advise adding a sentinel so
that you can test to see whether it's detected. if not, the problem is
in your RDNS config, nor in your zone generation.

--
P Vixie

when i generate an rpz i usually put in two rules for any known-bad domain:
example.com CNAME .
*.example.com CNAME .
this follows from my belief that the identifier was granted based on
public trust, which has not been met in this case, and it ought to be
revoked, but there's no mechanism for that, so we're doing this.
so if i knew of three bad names which shared a common parent, such as:
foo.example.com
bar.example.com
baz.example.com
i would generate the following RPZ content:
foo.example.com CNAME .
*.foo.example.com CNAME .
bar.example.com CNAME .
*.bar.example.com CNAME .
baz.example.com CNAME .
*.baz.example.com CNAME .
in no case would i connote irresponsibility to the parent, since it
might be an effective TLD and i would never want to block, e.g., *.com
or *.co.uk or *.co.nz or similar.

That's essentially the way the RRPZ file is generated, except that all explicitly blocked domains do NOT automatically merit a wild card, only those with explicitly blocked subdomains are reduced to a wild card. To be honest, I could generate the RRPZ file dramatically faster if I just automatically did a wild card for every blocked domain instead of checking if there are actually any explicitly blocked subdomains.
@ler762, you started this, what do you think? Should I just wild card all domains automatically and prune child domains that are already covered by a wild card? Like I said, I am loving that because it's faster for me to generate because there's less checks. But the question is does that satisfy the needs of all RPZ users? Many domains may not even have any subdomains, making wild cards useless but preventative nonetheless. Would these automatic wild cards just be excessive and blocking too much?

@vixie, please check the VRPZ files:
https://scripttiger.github.io/alts/
If we can all agree on what the best practice should be, I'll remove any competing formats.

Just to fully exhaust all possible versions of this, what about just the standard RPZ format with no subdomain pruning and just automatically adding a second wild card subdomain for every domain listed? Obviously this would increase file size, but I know from experience BIND can handle much larger files than that. Would this also still be usable? In the end it would have a lot of redundant entries, to be true, but would it still function for everyone? Right now the subdomain pruning itself is rather resource and time intensive due to the substring evaluations conducted on every entry when you add in that I am generating 16 different version of every available format, along with all the other data I generate not related to @StevenBlack. Processes that just do in-line text manipulations versus actual evaluations are obviously much faster, so if it functions and is as secure as possible, I would much rather go with that, even if it is not pretty and has a lot of redundancies.

@vixie, @ler762, @swapneelp, please check the BRPZ files:
https://scripttiger.github.io/alts/
Keeping in mind that security and functionality are my top priorities, not reducing redundancies only for the sake of reducing file sizes, do these files work for everyone? If so, I will remove the other RPZ versions and this will become the new "RPZ" format available with every release.
Being honest, we all know BIND can handle some massive zone file sizes and these are tiny compared to if you've ever worked with BIND in an enterprise environment. I have an open issue of my own related to this, https://github.com/ScriptTiger/Unified-Hosts-AutoUpdate/issues/25, and, as I said, I yield only to concerns of security or functionality, not file sizes.
I look forward to hearing everyone's thoughts on these topics!

I'd like it deduped but otherwise yes this is what i do and what i hope you do.
Get BlueMail for Android
On 24 Aug 2019, 21:57, at 21:57, ScriptTiger<EMAIL_ADDRESS>wrote:

Just to fully exhaust all possible versions of this, what about just
the standard RPZ format with no subdomain pruning and just
automatically adding a second wild card subdomain for every domain
listed? Obviously this would increase file size, but I know from
experience BIND can handle much larger files than that. Would this also
still be usable? In the end it would have a lot of redundant entries,
to be true, but would it still function for everyone? Right now the
subdomain pruning itself is rather resource and time intensive due to
the substring evaluations conducted on every entry when you add in that
I am generating 16 different version of every available format, along
with all the other data I generate not related to @StevenBlack.
Processes that just do in-line text manipulations versus actual
evaluations are obviously much faster, so if it functions and is as
secure as possible, I would much rather go with that, even if it is not
pretty and has a lot of redundancies.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/StevenBlack/hosts/issues/451#issuecomment-524600664

VRPZ it is! @ler762 won't have any complaints since it's blocking more. There will only be one RPZ link in the next update and the rest will be removed, so please update your links accordingly if you had anything configured to pull the discontinued demo formats.
Since this thread is out of scope anyway and related to DNS, I thought I'd also throw some questions I have out there to make sure a couple of my other formats are optimized. Please ignore my ignorance beforehand, I am actually a Windows user with experience in large enterprises using Windows products that are bound by support contracts and whatnot and I don't get to play with these other applications too often. That being said, I mostly just blindly generate data based on the specs people give me, but I would like to make sure a couple things are actually correct.
Right now my dnsmasq format uses the following format:
address=/zarget.com/<IP_ADDRESS>
Is this the best practice? I have seen some use the following:
address=/.zarget.com/<IP_ADDRESS>
Is the above simply to block all subdomains? Should both be used?
Also, my Unbound format currently uses the following:
local-zone: "zarget.com" redirect
local-data: "zarget.com A <IP_ADDRESS>"

Would this block zarget.com and all subdomains automatically with just these entries or is more needed?
Right now RPZ is the first format I have implemented pruning on since the idea of wild cards has popped up several times now and certainly does seem more secure, albeit will assuredly have its own mess of complications and broken websites because of it (i.e. the apple.com example given above). But nonetheless I would like to go ahead with implementing the pruning off of redundant subdomains from all formats used by software that can wild card.

Just finalized everything with the new RPZ format and reworked the de-duping/pruning so you won't have issues with subdomains overriding anything, but apple.com will be broken...I'm assuming no real loss there.
@ler762 mentioned Privoxy, so I threw in a new format for that, as well.
@StevenBlack mentioned Little Snitch earlier, and I actually did visit the links he provided, but maybe somebody could just post a quick sample here just to be sure.
And still looking for feedback for my previous inquiries on dnsmasq and Unbound, if anyone could just throw out their comments on those real quick, that would be much appreciated.

I forgot to link it in the previous comment, but just in case, those formats can once again be found here:
https://scripttiger.github.io/alts/

@ScriptTiger issue comments you create are editable...

The new RPZ list won't have any subdomains and I didn't make any compromises with it. @ler762 originally had an issue because subdomains of 2o7.net were being explicitly blocked, but I have corrected this in the latest version so absolutely no subdomains of blocked domains appear and no domain appears more than once on the list.
However, we encounter problems such as the following:
;<IP_ADDRESS> appleglobal.112.2o7.net #breaks apple.com
;<IP_ADDRESS> applestoreus.112.2o7.net #breaks apple.com

Because *.2o7.net blocks all subdomains and apple.com calls for the above subdomains explicitly, apple.com no longer works. It's not so much a problem with RPZ or my lists, it's more a problem with apple.com requiring people to link to untrusted domains.

@ScriptTiger Thanks for putting in the efforts/work. 👍
A suggestion w.r.t the file blacklist.txt - The file contains ^M ( carriage-return character ). It's no big deal. At my end, a dos2unix simply does the job.
I have configured a rpz zone in BIND with blacklist.txt. Will update  if I face any problems.

I just cURLed the list myself and it came out LF and no conversion was needed since cURL does that automatically, or at least the implementation of it that I am using does. What are you using to fetch your lists?
Right now all of the text files on the website are normalized to CRLF just to keep a running standard, since most of the users when I first started it were Windows users since they were the ones having most of the trouble and I was focused on improving accessibility for a wider range of folks.
I could easily normalize the RPZ files to LF if it's really an issue, I know Linux apps throw a fuss about that sometimes. For the most part Windows apps can read either LF or CRLF just fine, it's just visually when you throw it into some legacy text editors the line endings don't show up and everything gets squished together on one line, which has been an ongoing topic both in this repository and in mine from Windows users.
Like I said, it would be easy enough for me to normalize just those files, but breaking the standard could get confusing later on down the road in many respects since I myself use Windows due to occupational requirements, I generate the data on a Windows system using Windows scripts, and I would honestly just like to stay true to my original intent there to minimize confusion.

Evening @ScriptTiger in case you would like to provide the RPZ zones as *.txt format you have to ensure they end in LF (\n) as a lot of DNS servers in both Linux and windows will fail loading the files do to a mix of windows only (CRLF) vs *nix standard (LF)
So I suggest you run your list through either tr or dos2unix

Fine, fine. Since you all are ganging up on me about this, DONE! They are LF now. I let git take care of my line endings for me.

Just to finalize this thread, I have released a hosts file to RPZ converter for anyone running Windows that wants to convert their hosts files locally.
https://github.com/ScriptTiger/Hosts-Conversions
If you're wondering why it seems so slow, it's due to the fact that every subdomain must be broken into substrings for each higher domain and checked on an exact match basis in order to accurately identify it as a subdomain and remove it. Partial matching could speed things up considerably, but then it becomes less accurate when you consider many phishing domains stack domain names, such as XXX.com.YYY.com.
I have, however, made as many performance improvements as possible, such as skipping matching all together for second-level domains and automatically including them. This may become a problem if someone wants to block an entire top-level domain, but I have never seen anyone go that far before. And you can read the internal script remarks for more specifics if you're interested.
I would imagine if someone wanted to port this to a more efficient text-processing language, such as Perl, performance could be considerably improved again. However, I try to stick to the native Windows tools to lower the technical barriers that many people experience, so for now mine is just written as a standard batch file.

@vixie - I am curious, is there a home of the StevenBlack "hosts" file for RPZ?  I looked on the https://dnsrpz.info and did not a reference...  I did see the RPZ on the ScriptTiger site but it did not include the TTL or SOA lines.
Best regards, Jon


null safety

migration to null safety

Hey, thanks for the PR. Could you elaborate/explains your changes please?
I haven't used Dart in a while and I would like to understand the difference before I merge.

Hey, thanks for the PR. Could you elaborate/explains your changes please?
I haven't used Dart in a while and I would like to understand the difference before I merge.

I'm afraid I'm not good at explaining.
I recommend you to read Sound null safety | Dart, instead.
In addition to above, I additionally passed  optional arguments growable: false to named List constructors List.generate() and List.filled() replacing the unnamed constructor `List(), because the unnamed constructor generates fixed length lists, while named constructors generate growable lists by default which are very slow with subscripting.

I'm afraid I'm not good at explaining.
I recommend you to read Sound null safety | Dart, instead.
In addition to above, I additionally passed  optional arguments growable: false to named List constructors List.generate() and List.filled() replacing the unnamed constructor `List(), because the unnamed constructor generates fixed length lists, while named constructors generate growable lists by default which are very slow with subscripting.

I'm okay merging this, this will be released as a pre-release and breaking change as specified in the documentation.
I will also fix the linting issues and then publish a new version.
Thanks again for the pull request 👍

I'm okay merging this, this will be released as a pre-release and breaking change as specified in the documentation.
I will also fix the linting issues and then publish a new version.
Thanks again for the pull request 👍

https://pub.dev/packages/edit_distance/versions/0.5.0-nullsafety.0

https://pub.dev/packages/edit_distance/versions/0.5.0-nullsafety.0


V.2.0.4 Asset Browser missing many preview thumbnails

By my count, about 90 parts in the Asset Browser are missing preview thumbnails (also, some are displaying their object names rather than their in-game part names). Attached are some screenshots of representative tabs (note: I edited 2 of the attached JPEGs with red Xs to make some of the missing thumbnails easier to spot).

FWIW, I have may have a source for high-quality PNGs of many of the missing parts, if that's helpful. (I just don't know how to get them into the Asset Browser or else I'd fix it for myself.)

Should be sorted in latest versions.


Fix support for cookies bigger than 4KB in size in plugin-auth-node

Hey, I just made a Pull Request!
Browsers silently drop cookies that exceed 4KB, which can be problematic for refresh tokens and other large cookies. This pull request ensures that large cookies, like refresh tokens, are not dropped by browsers, maintaining the integrity of the authentication process. The changes include both the implementation of the cookie splitting logic and corresponding tests to validate the new functionality.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

This pull request is an alternative solution to a problem described in this pull request.

hey, @Rugvip @camilaibs - could we get an estimate when we could get feedback/review of this solution? Much appreciated.

Rather than slicing 4k characters. wouldn't it be better if we hashed it and the client had the hash as a lookup to the real refresh token?

@adminy

Rather than slicing 4k characters. wouldn't it be better if we hashed it and the client had the hash as a lookup to the real refresh token? Then local storage could store the refresh token.

It certainly would work - but in my opinion it's better to store refresh token (chunked or not) as an HttpOnly cookie as it is safer (localStorage can be accessed via javascript (i.e. XSS) - but HttpOnly cookie cannot). It also would require a lot more changes to achieve that (frontend auth code or perhaps the whole current auth flow would need to be modified) - what I'm aiming to do with this PR is to produce a small fix for situation that we've encountered in our organization (currently we had to fork entire plugin-auth-node package to introduce similar fix - it's not a feasible long term approach).

your PR looks great to me! I hope someone on the backstage team takes a look soon.

Hey @Rugvip @camilaibs! Could we get some rough estimate on when we can expect some actions from your side on this pull request? While this is rather critical for us and our organisation, we would highly appreciate any input so we can plan future work and potentially move forward with our project.
Thanks in advance!

@benjdlambert could you take a look? :)

For anyone following along: We had a chat on discord about some of the details here. It's unfortunate that the upstream service in this case is causing undesired complexity by generating "unreasonably" (for some definition of the word) large tokens. But, it's also true to say that the Backstage framework chose to put third party data in cookies instead of a separate session store, and didn't handle scaling up to larger data sizes because of that choice. So us handling those larger data points for any reason (for example a large number of custom scopes being granted, or so), is not necessarily a bad thing. It's an internal concern of ours, and it has a bug/limitation. So we'll have to choose whether we accept to have that limitation and cut off some integration abilities (to save on complexity), or to accept this change, or switch to a different less limited session data store.
Added to the board for discussion with the team.

anyone?

Hey backstage team! @Rugvip @camilaibs @freben , while we are struggling with forked auth-node plugin for almost 10 weeks now, it seems like there has been no progress in this PR. Could you please take a moment to review those changes? They make absolutely no harm to the existing ecosystem, but make it viable for all users to authenticate with no issues :)

To build on what @bsiedleckiallegro wrote: even if you choose to switch to a different (less limited) session data store in the future (as per @freben comment) changes in this PR are (in my opinion) acceptable stopgap solution for fixing some of edge cases. And while it's true that they introduce a bit of additional complexity - for 99% cases this implementation behaves exactly like old one would.


Better workaround for silent token refresh failure in Safari

Library

[x]<EMAIL_ADDRESS>or<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>
Description
Current work around for silent token renewal failure in safari requires user intervention i.e to completely disable the third party cookie blocking in Safari Preferences.However it appears a better alternative exists in the form of Storage Access API.
From  MDN doc

In addition, sandboxed s cannot be granted storage access by default for security reasons. The API therefore also adds the allow-storage-access-by-user-activation sandbox token. The embedding website needs to add this to allow storage access requests to be successful, along with allow-scripts and allow-same-origin to allow it to call the API, and execute in an origin that can have cookies

From webkit blog

To keep supporting cross-site integration, we shipped the Storage Access API two years ago to provide the means for authenticated embeds to get cookie access with mandatory user control.

Is this something we can look into as a possible alternative ?

@niwsa Thanks for posting this. We have looked at the Storage API and we are currently exploring options that will require changes to both the auth client library and server.
The current workaround we are using is to introduce the authorization code flow for browsers, which limits the usage of iframes. We have introduced this in the @azure/msal-browser package which is currently in beta. We recommend using this package to get token exchange working in Safari browsers.

@pkanher617 Thanks for the reply.However I understand that @azure/msal-browser is not available for B2C tenants which is really my use case.For now thinking of using https://github.com/AzureAD/passport-azure-ad with server side rendering.

@niwsa We are hoping to have support for B2C tenants by the end of this month, keep an eye out for new releases!

@pkanher617 - re your comment on 14th May - are B2C tenants now supported please?


use single quotes for pip install commands with square brackets

Related to #402 and #412 .
pip install -e dvc[all] fails when ZSH is being used
pip install -e 'dvc[all]' should be working everywhere sh, bash, zsh.
Check that it's actually working and update our docs.

pip install -e '.[all,tests]' works for me on zsh

Can I be assigned this issue @shcheklein ?

Can I be assigned this issue @shcheklein ?

@Naba7 sure! thanks. feel free to fix it.

Hey @shcheklein Its really nice to say thanks but I kind of like DVC and want to explore and know about the stuff happening around,so there is no need to say thanks.

@shcheklein

pip install -e '.[all,tests]' works for me on zsh.

I think this can be closed.

@algomaster99  ... and update our docs - that's the primary goal of this ticket :)
We need to go put [something] into single quotes.

@shcheklein why do we need to put it in single quotes if it is working without them? The docs already don't include them in signle quotes.

@algomaster99 it does not work w/o them on ZSH

@shcheklein Okay, I'll make the changes but test it once again just to be sure. :)


Favicon being loaded from WSO2's website

Description:
When directly access the API from the web browser our backend response seems to load a favicon.ico from the wso2.com website. Customers requestes to remove or disable this favicon loading location header which is coming in the response of the API call through a web browser.
Steps to reproduce:
Affected Product Version:
APIM 2.2

Fixed with https://github.com/wso2-support/carbon-mediation/pull/470


[WIP][ci][python] updated pep8 to pycodestyle

pep8 is now pycodestyle.
Refer to https://github.com/PyCQA/pycodestyle/issues/466 .
This PR removes the following warning:

New version finds some warnings, which should be fixed.

Added W503 into ignore list according to https://github.com/PyCQA/pycodestyle/issues/498#issuecomment-331454473 .
Also I've excluded .nuget folder from checks because the line       <file src="lib\**" target="lib"/> yields "W605 invalid escape sequence '*'". Is there any way to escape * without breaking the functionality?


Fix category icons getting cropped in torrent-preview

Coverage remained the same at 24.341% when pulling b8fb31fe8c250b5597fd259ccc7fd78aaeb4e909 on Kiloutre:patch-17 into 00e092372f8339bc8f7fd135fb4fc0784403ce71 on NyaaPantsu:dev.

Coverage remained the same at 24.341% when pulling 4f2a72061f7978c02a39c13a746c92735a6080b4 on Kiloutre:patch-17 into c2541bed3e99c923e28a1da12fa5f6135e1e2e32 on NyaaPantsu:dev.


Capital C Credits

Audit all copy for this

This Special Task From Mr @kauffj Was Finished By Chris From Indonesia.

#1407

thank you @ykris45!

thank you @ykris45!

You Welcome sir. Memeking always Help for LBRY Universe


Native encoding/decoding changes for interceptors

Fixes #1109

Disable outbound content type interceptor when nativeEndoding is enabled on the producer

Disable inbound content type interceptor when nativeDecoding is enabled on the consumer

Add a new flag in ConsumerProperties for useNativeDecoding

Apply ContentType interceptor before the InboundInterceptor to ensure
that content type set on the channel is applied on the message if content type
is missing on the Message.

Add tests for verifying interceptors are not applied when native encoding/decoding is enabled

Fix a kryo integration test

Remove Inbound interceptor from MessageCollector when native encoding is enabled on the producer

Polishing

Codecov Report

Merging #1133 into master will decrease coverage by 0.13%.
The diff coverage is 53.84%.

@@             Coverage Diff              @@
##             master    #1133      +/-   ##
============================================
- Coverage     56.64%   56.51%   -0.14%
- Complexity      662      667       +5
============================================
  Files           121      121
  Lines          3425     3433       +8
  Branches        500      503       +3
============================================
  Hits           1940     1940
- Misses         1317     1320       +3
- Partials        168      173       +5

Impacted Files
Coverage Δ
Complexity Δ

...mework/cloud/stream/binder/ConsumerProperties.java
64.1% <50%> (-1.62%)
13 <1> (+1)

...oud/stream/binding/MessageConverterConfigurer.java
60.27% <50%> (-2.67%)
20 <0> (+4)

...rk/cloud/stream/test/binder/TestSupportBinder.java
90.62% <66.66%> (-2.93%)
4 <1> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9cc0b83...1b29aa5. Read the comment docs.

@sobychacko I'll address that one comment and will merge


Add Translations for zh-rCN

Add Translations for zh-rCN

Looks good, just one little thing that needs fixed before merge.

OK.

Fixed Translations for zh-rCN
Add Translations for zh-rTW,zh-rHK

I squashed these into a single commit and merged to master. Thanks!


tcptrace: init at 6.6.7

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@Twey are you able to continue with this PR and address the issues mentioned?

Please feel free to reopen this request if you are interested in continuing.


Setting timeout: 0 in waitForFunction does not remove default timeout condition

What steps will reproduce the problem?
Please include code that reproduces the issue.
await page.waitForFunction(
    () => document.querySelector('.loading-item').getAttribute('something') === 'takes a long time',
    { timeout: 0 })

What is the expected result?
That waitForFunction will continue waiting without limit
What happens instead?
I get the following error:

Error: waiting failed: timeout 30000ms exceeded at Timeout.WaitTask._timeoutTimer.setTimeout

Thoughts
Maybe this is by design? Perhaps it is bad practice to query the page without a timeout in place, but my use case is file uploads, wherein I have no idea the network speed, so I cannot calculate a reasonable timeout.
Moreover, page.waitForNavigation has a timeout override.
Any other thoughts here?

This has been fixed in https://github.com/GoogleChrome/puppeteer/pull/1964. Will be available in the next release.


Possible false positives with aws-access-token rule

Describe the bug
The aws-access-token rule detects IAM resource IDs.
The regex contains these prefixes: AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA
Those prefixes are associated with IAM resource IDs, which I believe are not actually sensitive info.
https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html#:~:text=Understanding unique ID prefixes

To Reproduce
Steps to reproduce the behavior:

In AWS, go to IAM and create a new Role
Copy the name of your role.
Use the aws CLI to get your role:

aws iam get-role --role-name TestRole

{
    "Role": {
        "Path": "/",
        "RoleName": "TestRole",
        "RoleId": "AROAWORVRXQ5NC76T7223",
        "CreateDate": "2022-12-01T12:59:06+00:00",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                    },
                    "Action": "sts:AssumeRole",
                    "Condition": {}
                }
            ]
        },
        "Description": "Delete this after testing",
        "MaxSessionDuration": 3600,
        "RoleLastUsed": {}
    }
}

Copy the RoleId and add it to a file in your repo
Run gitleaks. e.g. gitleaks protect --verbose --staged

Expected behavior
I believe Gitleaks should not detect the RoleId as a secret, but it does.
Finding:     "RoleId": "AROAWORVRXQ5NC76T7223",
Secret:      AROAWORVRXQ5NC76T722
RuleID:      aws-access-token
Entropy:     3.684184
...

Also, interestingly, the RoleId is 17 characters after the "AROA" prefix. The aws-access-token regex only detects the first 16 characters. Is that intentional?
Screenshots

Basic Info (please complete the following information):

OS: Windows 10 Home (19044.2251)
Gitleaks Version: 8.15.1

Additional context
N/A
cc @zricethezav

We are also facing this issue in our repositories. There are some IAM Instance Profile and IAM Role definitions that (as far as I know) are not sensitive information after all.
We suppressed these findings globally by adjusting the aws-access-token rule using a custom gitleaks.toml:
title = "Custom gitleaks config"

[extend]
useDefault = true

[allowlist]
description = "Custom global allow lists"
paths = [
    '''gitleaks.toml''',
    '''(.*?)(jpg|gif|doc|docx|zip|xls|pdf|bin|svg|socket)$''',
    '''(go.mod|go.sum)$''',
    '''node_modules''',
    '''package-lock.json''',
    '''vendor''',
    '''gitleaks-allow''',
]

#######################
# Customized rules, based on https://github.com/zricethezav/gitleaks/blob/v8.16.0/config/gitleaks.toml
#######################

# Customizations:
# - Remove 'AGPA|AIDA|AROA|AIPA|ANPA|ANVA' to reduce false-positive findings
[[rules]]
description = "AWS"
id = "aws-access-token"
regex = '''(A3T[A-Z0-9]|AKIA|ASIA)[A-Z0-9]{16}'''
keywords = [
    "akia","asia",
]

This way we only get findings for Access Keys and STS Tokens.
If you think that this should be changed, I can create an pull request for that.
Regards
Malte


Simplification of sample input into config files

The requirement to input the same sample names in multiple places is confusing and potentially error-prone. It would be much better to have one location where samples are named.

That is a good point, and @cgrahlm will look into how we can reduce redundancy in those sample name inputs.

This has been fixed in v1.2, so sample names only need to be entered one time within the config file.


add_items() with deleted element id keeps element marked deleted

import hnswlib
import random

dim = 4
objects = []
for i in range(2) :
    vector = random.sample(range(1000), dim)
    objects.append(vector)
print(objects)
index = hnswlib.Index(space='l2', dim=dim)
index.init_index(max_elements=5, ef_construction=200, M=32)
index.add_items(objects,[12,3])
print(index.get_ids_list())
print(index.get_items([3]))
index.mark_deleted(3)

try:
    print(index.get_items([3]))
except RuntimeError:
    print("el not found")

index.add_items(objects[1],3)

try:
    print(index.get_items([3]))
except RuntimeError:
    print("el not found")

print(index.get_ids_list())

[[145, 640, 35, 805], [633, 104, 726, 950]]
[12, 3]
[[633.0, 104.0, 726.0, 950.0]]
el not found
el not found
[12, 3]

@qwertyforce Thanks for reporting! @apoorv-sharma seems like we missed a bug

Thank you. Looks like two fixes needs to be done (independent of each other):

If element is deleted, and it is added back again unmarkDeletedInternal should be called before starting any update operation or an error could be thrown. But unmarkDeletedInternal is not exposed to the users yet to unmark on their own, so former approach would be better.
Fix get_ids_list  method to exclude deleted elements in general.

@yurymalkov Let me know if it sounds good, i will fix the 1) soon.

@apoorv-sharma 1) Sounds great! Thank you!
I think for 2) we probably need an option to include both (e.g. with a flag like return_deleted), otherwise there probably will be no option to access the ids of the deleted elements from python.


can i put own google sheet?

i want to compile with my own categories and my own lists, that is possible?

No


I get window undefined error when using web3react demo on nextjs app

I get this error when I use the stargazer web3react connector
#ReferenceError: window is not defined
at new StargazerConnector (/home/node_modules/@stardust-collective/web3-react-stargazer-connector/dist/cjs/connector.js:32:9)
at eval (webpack-internal:///./connectors/stargazer.ts:8:28)
at Object../connectors/stargazer.ts (/home/.next/server/pages/_app.js:660:1)
at webpack_require (/home/.next/server/webpack-runtime.js:33:42)
at eval (webpack-internal:///./utils/getWalletProviders.tsx:16:79)
at Object../utils/getWalletProviders.tsx (/home/.next/server/pages/_app.js:1563:1)
at webpack_require (/home/.next/server/webpack-runtime.js:33:42)
at eval (webpack-internal:///./components/layouts/GlobalLayout/Navbar/Profile/ProfileWalletConnection.tsx:11:82)
at Function.webpack_require.a (/home/.next/server/webpack-runtime.js:97:13)
at eval (webpack-internal:///./components/layouts/GlobalLayout/Navbar/Profile/ProfileWalletConnection.tsx:1:21) {
page: '/'
}
This is how I declare the connector
#import { StargazerConnector } from '@stardust-collective/web3-react-stargazer-connector';
declare global {
interface Window {
stargazerConnector: StargazerConnector;
}
}
const stargazerConnector = new StargazerConnector({
supportedChainIds: [1, 3]
});
window.stargazerConnector = stargazerConnector;
export { stargazerConnector };

What platform are you using?, the window object should be available in most contexts

@petermunachi Since NextJS runs code on both the front end and back end, you may need to ensure that the StargazerConnector is only running on the front end code.
This StackOverflow answer might help:
https://stackoverflow.com/questions/55151041/window-is-not-defined-in-next-js-react-app


snap-server <IP_ADDRESS>

snap-server-<IP_ADDRESS> is out of bounds for:

[ ] http-streams-<IP_ADDRESS> (==1.0.*). Andrew Cowie<EMAIL_ADDRESS>@afcowie. @afcowie. Used by: test-suite
[ ] websockets-snap-<IP_ADDRESS> (==1.0.*). Jasper Van der Jeugt @jaspervdj. @jaspervdj. Used by: library

No upper bound will be put in place in this case. The test suite for http-streams will remain skipped, and websockets-snap will remain blocked, until they are compatible with this version of snap-server.

Just tested http-streams using nightly-2018-05-02 and thus GHC 8.4.2; all good.
@DanBurton not entirely sure what I'm supposed to do here, other than to report it's ok?
AfC


crowdsec: 1.4.3 -> 1.4.4

Diff: https://github.com/crowdsecurity/crowdsec/compare/v1.4.3...v1.4.4
Changelog: https://github.com/crowdsecurity/crowdsec/releases/tag/v1.4.4
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.


Atomic Symbol Generator

Aim
To make a simple project using HTML, CSS, and Javascript to fetch info from a form and manipulate the data into a good-looking atomic symbol. It will encourage the use of javascript in static sites and also give rise to a cool output.
Details

The user would need to enter info like their Name, Age and Weight
The input would be used to form a good looking atomic symbol
The weight will be treated as atomic weight and the age as atomic number
Electronic configuration will also be displayed.

sample photos:

Under which program you can contribute mention here

I want to contribute for the Hacktoberfest. I can create a new folder named Atomic Symbol Generator for this project.

Go Ahead @yash22arora

Created a PR, Please review it and merge if it's okay! It would be great if you could also give the "hacktoberfest-accepted" label to the PR


More granular organization for model files

Is your feature request related to a problem? Please describe.

ilab handles multiple types of model files throughout a typical workflow:

Downloaded models
Intermediate checkpoints
Finished, trained models

For a long time, ilab used a single directory to handle all these files. Recently, with @RobotSail's dedicated storage PR (https://github.com/instructlab/instructlab/pull/1471), downloaded models got separated out into their own directory. This highlighted some potential issues, namely https://github.com/instructlab/instructlab/issues/1705.
Working on that issue brought into relief the fact that we handle several different types of model files, and that they should be stored with different levels of ephemerality depending on their type.
Describe the solution you'd like

I think we should adhere to the following pattern:

Downloaded models should live in either cache or persistent user storage (there are good arguments for both alternatives)
Intermediate checkpoints should live in cache (they are inherently temporary)
Finished, trained models should live in persistent user storage (they are unique user data that should never be automatically removed)

Additional context

Brought about as a related issue to https://github.com/instructlab/instructlab/pull/1762.

@tiran @russellb Thoughts?

@cdoern No, I just filed it as a follow-up to #1762. It's unclear if this is still relevant or needed - the code may have changed since I filed this.


Use '/usr/bin/env sh' instead of '/bin/sh'

so that the maker runs on nixos. Should be more universal.

Thanks!


Fix Geoblacklight document generation

Closes #890

Looks good, the slug spec is now a lot cleaner too.


It doesn't seem to work properly

I can't find it in ob？The morning is normal。

it is ok now 🌹

I get it why!
It can be found by editing the state, but not by previewing the state. Ha ha
😂   😂   😂


[BUG] Cannot Provision RKE2 v1.24.15-rc4+rke2r1 on v1.2.0-rc2 w/ v2.7-head

Describe the bug
Harvester v1.2.0-rc2 seems to not be able to provision v1.24.15-rc2+rke2r1 w/ Rancher v2.7-head & has also issues with Rancher v2.7.5-rc5
To Reproduce
For v2.7.5-rc5:

Install v2.7.5-rc5 Rancher & Single Node v1.2.0-rc2
Modify RKE Metadata in Rancher Settings to: https://raw.githubusercontent.com/rancher/kontainer-driver-metadata/rke2-2023-june/data/data.json
Refresh RKE Metadata via Cluster Management -> Drivers in Rancher
Ensure Harvester is imported into Rancher
Create cloud credential
Either create single or multi machine RKE2 cluster at v1.24.15-rc2+rke2r1
Note that Cloud Provider Driver is v0.1.1400, but for validating https://github.com/rancher/rke2/issues/4184 , we'd need to use more recent Rancher, not rc5?

For v2.7-head Rancher:

Install v2.7.5-rc5 Rancher & Single Node v1.2.0-rc2
Modify RKE Metadata in Rancher Settings to: https://raw.githubusercontent.com/rancher/kontainer-driver-metadata/rke2-2023-june/data/data.json
Refresh RKE Metadata via Cluster Management -> Drivers in Rancher
Ensure Harvester is imported into Rancher
Create cloud credential
Either create single or multi machine RKE2 cluster at v1.24.15-rc2+rke2r1
Should display "Error applying plan"

Note,
Testing with 22.04 Ubuntu Jammy CloudImage & OpenSUSE Leap 15.04 CloudImage
Expected behavior
The cluster to provision
Environment

Harvester ISO version: v1.2.0-rc2
Rancher v2.7.5-rc5 && v2.7-head
Underlying Infrastructure: bare-metal

Additional context
Seeing logs like:
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=warning msg="Failed to get image from endpoint: GET https://index.docker.io/v2/rancher/system-agent-installer-rke2/manifests/v1.24.15-rc2-rke2r1: MANIFEST_UNKNOWN: manifest unknown; unknown tag=v1.24.15-rc2-rke2r1"
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=error msg="error while staging: all endpoints failed: GET https://index.docker.io/v2/rancher/system-agent-installer-rke2/manifests/v1.24.15-rc2-rke2r1: MANIFEST_UNKNOWN: manifest unknown; unknown tag=v1.24.15-rc2-rke2r1: failed to get image index.docker.io/rancher/system-agent-installer-rke2:v1.24.15-rc2-rke2r1"
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=error msg="error executing instruction 0: all endpoints failed: GET https://index.docker.io/v2/rancher/system-agent-installer-rke2/manifests/v1.24.15-rc2-rke2r1: MANIFEST_UNKNOWN: manifest unknown; unknown tag=v1.24.15-rc2-rke2r1: failed to get image index.docker.io/rancher/system-agent-installer-rke2:v1.24.15-rc2-rke2r1"

And like:
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=info msg="[Applyinator] No image provided, creating empty working directory /var/lib/rancher/agent/work/20230621-205307/9e67e0f44521fd2eabd29a6efd906e66aebf98b802c6dc21d401bf8bcc84a3b9_0"
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=info msg="[Applyinator] Running command: sh [-c rke2 etcd-snapshot list --etcd-s3=false 2>/dev/null]"
Jun 21 20:53:07 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:07Z" level=info msg="[Applyinator] Command sh [-c rke2 etcd-snapshot list --etcd-s3=false 2>/dev/null] finished with err: <nil> and exit code: 127"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error loading x509 client cert/key for probe kube-apiserver (/var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt//var/lib/rancher/rke2/server/tls/client-kube-apiserver.key): open /var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt: no such file or directory"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error loading CA cert for probe (kube-apiserver) /var/lib/rancher/rke2/server/tls/server-ca.crt: open /var/lib/rancher/rke2/server/tls/server-ca.crt: no such file or directory"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error while appending ca cert to pool for probe kube-apiserver"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error loading CA cert for probe (kube-controller-manager) /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: open /var/lib/rancher/rke2/server/tls/kube-controller-manager/kube-controller-manager.crt: no such file or directory"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error while appending ca cert to pool for probe kube-controller-manager"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error loading CA cert for probe (kube-scheduler) /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: open /var/lib/rancher/rke2/server/tls/kube-scheduler/kube-scheduler.crt: no such file or directory"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error while appending ca cert to pool for probe kube-scheduler"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=info msg="[K8s] updated plan secret fleet-default/testv124test-bootstrap-template-fbgqb-machine-plan with feedback"
Jun 21 20:53:08 testv124test-pool1-febebd10-p5dfx rancher-system-agent[1865]: time="2023-06-21T20:53:08Z" level=error msg="error encountered during parsing of last run time: parsing time \"\" as \"Mon Jan _2 15:04:05 MST 2006\": cannot parse \"\" as \"Mon\""

Via the journalctl journalctl -u rancher-system-agent.service --follow - when shelled into the VMs that are providing the RKE2 on Harvester.

The metadata seems to have it:

╭─mike at suse-workstation-team-harvester in ~/Projects/rancher-cleanup on main✔
╰─± curl https://raw.githubusercontent.com/rancher/kontainer-driver-metadata/rke2-2023-june/data/data.json | jq '.rke2.releases[] | select( .charts["harvester-cloud-provider"]["version"] == "0.2.200" ) | {"rke2-releases-rke2-version": .version, "rke2-releases-rke2-provided-harvester-cloud-provider": .charts["harvester-cloud-provider"]}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 5994k  100 5994k    0     0  26.9M      0 --:--:-- --:--:-- --:--:-- 26.9M
{
  "rke2-releases-rke2-version": "v1.24.15-rc4+rke2r1",
  "rke2-releases-rke2-provided-harvester-cloud-provider": {
    "repo": "rancher-rke2-charts",
    "version": "0.2.200"
  }
}
{
  "rke2-releases-rke2-version": "v1.25.11-rc4+rke2r1",
  "rke2-releases-rke2-provided-harvester-cloud-provider": {
    "repo": "rancher-rke2-charts",
    "version": "0.2.200"
  }
}
{
  "rke2-releases-rke2-version": "v1.26.6-rc4+rke2r1",
  "rke2-releases-rke2-provided-harvester-cloud-provider": {
    "repo": "rancher-rke2-charts",
    "version": "0.2.200"
  }
}

Closing - as v2.7.5-rc5 seems to be able to see it now at v0.2.200

I think it was just a slight metadata refresh issue on the env...


Top-level schemas are not resolved when assigned to resources

For example:
#%RAML 0.8
title: Example Web API
version: v1
schemas:
  - Thingy: !include includes/post-thingy-schema.json
# <---snip--->
      responses:
        200:
          body:
            application/json:
              schema: Thingy

The last line, when parsed, isn't actually resolved to the defined schema:
>>> resp = endpoint.responses[0]
>>> resp.body[0].schema
'Thingy'

It should behave like endpoint.resource_type were you can actually access the schema.

I hit the same problem, instead of getting the schema I got name of it.
I was thinking, if that was by design or not. The documentation: https://ramlfications.readthedocs.org/en/latest/api.html#ramlfications.ramlfications.parameters.Body does not expect a name of the schema to be expected result.
To me it really seems more practical to get the schema resolved (as described in the original PR).

Thanks @vlcinsky - I am working on a couple of patches related to this and #23 - good to know there are some users of the library that want this functionality :D I agree that resolving the schema is ideal.

@econchick Sounds very promising. I am really found of RAML (after longer fight with Swagger I have to appreciate, RAML serves much better my testing needs and has much better core concepts). Coding most of my stuff in Python I found ramlfications of great value. Thanks you for that.

What is the status of this issue? It is already deprecated in RAML 1.0 to name it schemas and use types instead. But the issue will be the same: a name instead of an schema object.
Do you use definitions to validate an input or output? And how do you do it? I am using jsonschema for now, but I thought it would be nice feature of ramlfication to validate object against defined schema.

@rebeling can you link me to where RAML 1.0 explicitly says schemas are deprecated? I may have just missed it.
My intention with schemas (related: #15) is to indeed have ramlfications validate against the defined schema. Same with the newly added Data Types.
I'm putting a lot of work on this library this weekend (see roadmap), hopefully I can address this too.

#15 ok, missed this one - yeah, jsonschema ;)
For deprecation for schemas find the terms at http://blog.raml.org/. I am ramlficated already from your project, thanks and stay tuned.

What is the status of this issue?

@pokidovea hmm yeah - 0.2.0 development is a bit slow. How much of a blocker is this particular issue for you?

Unfortunately, it prevents me from using this lib. I have to use JS analogue.

@pokidovea ah well, we don't want that! 😄  I'll try to get some time this week/weekend to address this.


SW-6856 Move tests to build.yml and add tag version job

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 78.28%. Comparing base (0e820ea) to head (56eebca).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #39   +/-   ##
=======================================
  Coverage   78.28%   78.28%
=======================================
  Files           7        7
  Lines         152      152
=======================================
  Hits          119      119
  Misses         30       30
  Partials        3        3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


make StableTasks macros tab-completable

This is one option to close #56.

Although a bit hacky, I like it :)

chunks and TaskLocalValue are missing


[BUG] Investigate Missing Source Link & Untracked Sources Issues

Investigate why Source Link entry in signed nuget package (When viewed with the package explorer) shows up as Missing Symbols for dev builds, but show up as Contains Untracked Sources (obj) for non-dev (Manual) builds. Also what part of the signing process affect the symbols.

In this ServiceBus build why did Azure.Messaging.ServiceBus have missing symbols while Microsoft.Azure.ServiceBus symbols are not missing.

Investigate the issues of untracked sources. How does it relate to This PR

According to the tool tips over the warning and error icons:

Source Link: we should either add <EmbedUntrackedSources>true</EmbedUntrackedSources> to our project or use the workaround in dotnet/sourcelink#572. Given we use Source Link already, I thought, I'm not sure why we're getting this. Maybe it's just precautionary to make sure packages are built with full sources either way?
Deterministic (dll/exe): if we're using the .NET 2.1.300 SDK or newer, we should add <ContinuousIntegrationBuild>true</ContinuousIntegrationBuild> to our project.
Compiler Flags: seems to indicate we should use the .NET 3.1 compiler targeting netstandard2.0.

Closing this since all the info is here https://github.com/Azure/azure-sdk-for-net/issues/12995
Will continue to use https://github.com/Azure/azure-sdk-for-net/issues/12995 to track the issue till its all resolved.
Thanks @heaths


Color Identifier

Is your feature request related to a problem? Please describe
A GUI application which can browse an image and tell you color name and HEX code for the cursor position/pin. GUI with random themes
Describe the solution you'd like
a GUI application

Please assign me this one.
Thank you

@Mysterious-Owl please start working. I am having a mac issue after the new macOs 12.6 update. I will assign it to you as soon as my laptop is fixed.
Also if you are done don't hesitate to open a PR


bug: attempt to index field 'uv' (a nil value)

Did you check docs and existing issues?

[X] I have read all the lazy.nvim docs
[X] I have searched the existing issues of lazy.nvim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
v0.9.5
Operating system/version
Arch Linux 6.8.1
Describe the bug

ERROR Failed to run healthcheck for "lazy" plugin. Exception:
function health#check, line 25
Vim(eval):E5108: Error executing lua ...ain/.local/share/nvim/lazy/lazy.nvim/lua/lazy/health.lua:29: attempt to index field 'uv' (a nil value)

Steps To Reproduce

Update to v10.18.0
Run :checkhealth

Expected Behavior
:checkhealth run w/o exception.
Repro
No response

Ia already fixed and released

any steps on how to reinstall lazy ? it wont start because of the broken version now

To reinstall fixed version, back up your local files
mv ~/.local/share/nvim{,.bak}
mv ~/.local/state/nvim{,.bak}
mv ~/.cache/nvim{,.bak}

And then, rerun nvim

any steps on how to reinstall lazy ? it wont start because of the broken version now

I fix it in this way.
edit  ~/.local/share/nvim/lazy/lazy.nvim/lua/lazy/inti.lua+5
modify "vim.uv = vim.uv or vim.uv" to "vim.uv = vim.uv or vim.loop"
then you can enter nvim normally and reinstall lazy.nvim by its own ui

For others running into this, just remove lazy and let it re-install itself.
rm -rf ~/.local/share/nvim/lazy/lazy.nvim

I'm actually still having this problem. Fresh install. If I check .local/share/nvim/lazy/lazy.nvim/lua/lazy/core/config.lua I see M.version = "10.19.0".

I ended up checkout out version 10.17.0, and things work as expected. Are we sure the fix worked?

Yes, unless require('lazy') is not the first lazy module you require.

@tylercecil if you're using LazyVim, then you also need to update that

I just copied the install to a fresh init.lua, using nvim appimage v0.9.5 with no configuration and I get the error. To resolve it I had to swap uv with loop (found in an older tutorial) and it worked. I could then switch back to uv without issue. It seems like it is still an issue for new users.

I also get this issue.
I've modified my installation to reflect the latest README with or
if not (vim.uv or vim.loop).fs_stat(lazypath) then

my ~/.config/nvim/init.lua has
require("lazy").setup(plugins, opts)

And have deleted my ~/.local/share/nvim/lazy directory and started nvim again.
Running :checkhealth I see
lazy: require("lazy.health").check()

- ERROR Failed to run healthcheck for "lazy" plugin. Exception:
  function health#check, line 25
  Vim(eval):E5108: Error executing lua ...ide/.local/share/nvim/lazy/lazy.nvim/lua/lazy/health.lua:29: attempt to index field 'uv' (a nil value)
  stack traceback:
  ...ide/.local/share/nvim/lazy/lazy.nvim/lua/lazy/health.lua:29: in function 'check'
  [string "luaeval()"]:1: in main chunk

Checking
~/.local/share/nvim/lazy/lazy.nvim/lua/lazy/core/config.lua | grep 'M.version'
M.version = "10.20.2" -- x-release-please-version

my nvim is 0.9.5 as well (from Arch repo)

That's because you used vim.uv in the bootstrap code.
The readme has been updated to correct this
https://github.com/folke/lazy.nvim?tab=readme-ov-file#-installation

I'm encountering the same problem. I installed neovim from AUR(0.9.5). Only the bootstrap code is in init.lua.


hellos have a question

How to input a 2D image to keras?
I am using open cv2 to input four image then combine them into one,
now  I want to do is input my new image into keras,how to do that?


WIP: Test runc main

This is just a draft PR to verify that upgrading runc won't break the tests, as it did in the past. It should be fixed in this version, let's verify it!

@AkihiroSuda can you please add the ok-to-test label? I think that kicks some k8s test

/ok-to-test

/retest
heh, something else is failing to init those.

/retest

/retest

/retest

/retest

/retest


RFC/WIP: adopt ruff and its code upgrade/format methods

this starts with the isort and pyugprade rules + flake8 based rules
some warnings are resolved - about 141 more to go
i stumbled across a number of leftovers from historic python2 support
i'd like to get some input on whether adding type annotations would be appreciated or frowned upon
i'd like to get input on whether this type of project modernization would be appreciated or not before digging into the leftover linter warnings

I haven't looked through all of it, but I give it thumbs up.  I should probably modernize my python skills myself one of those days :-)

lovely, i'll gradually work trough the rest then as i want to use the library for modification of caldav calendars and want to give back

... but I should add, I find backward compatibility very important, as well as support for old python versions.  I even believe it's a nice thing to support python versions that are already officially "EOL", as those python versions often are still included in distros that are not EOL.  Though, python2 is officially not supported anymore, and there was a pull request removing python 3.6 from the test matrix slipping through some weeks ago :-)

While personally I prefer to shed dead snakes as dropping support code and adopting new features tends to clean up and Speed up code, I'm absolutely on board with keeping them for projects that consider them important
In those cases I strongly appreciate a definition of the intended support range in order to avoid endless support for dead snakes (I love the win some of the newer versions enable)

Here are my thoughts:

As a bare minimum, all python versions that are officially supported by upstream should be supported
Shedding of old python version should be well announced in the CHANGELOG.md file
Preferably, shedding of old python versions should only be done when changing the major version number of the library (I did have great plans for the 2.0-version of this library, but perhaps I will need to procrastinate it until 8.0 or something) :-)
It's a nice thing to keep an overview of python versions used by the popular Linux distros that aren't EOL'ed yet, and consider that before shedding support for old python versions
We try to avoid "schrödingers support". It's a nice thing to toss a warning or error if the python version is not officially supported, rather than something suddenly breaking out of the blue in a production environment.

So this will go into a 2.0-release, which will officially break with 3.7 (and possibly 3.8 as well).  I cannot give any ETA on a 2.0-release currently, but possibly in some few months.

Now that 1.4.0 has been released, we may consider shedding support for python 3.7 and python 3.8 and work towards a 2.0-release.
There are references to python 3.7 in docs/source/index.rst, pyproject.toml and .github/workflows/tests.yaml.  If those are fixed, I believe the tests will pass, and this work may be merged into main.

i started picking this up again - there are still dozens of warnings, some dead code and more

im particularly confused around the numbered variables in tests, of which some are unused. some are missed all together

im particularly confused around the numbered variables in tests, of which some are unused. some are missed all together

The functional tests are more or less a complete mess, developed organically over time.  I've considered doing something with it, but ... it's not a high priority.  An idea could be to rename it into test_legacy_ftest.py or something, and then make new test code from scratch (and deleting the legacy tests once the new tests are well enough done).
One of the problems with the tests is that many of them are not really testing the library code per se, but rather the servers.  I started today writing up some new code in the test directory, check_server_compatibility.py, I will push it in a separate branch as soon as it's doing anything useful.  The purpose is to make a tool that can be run from the command line towards any server and spit out a list of "quirks".  The tests today work the other way around, one have to provide the expected "quirks" and then the tests will break if the list is incorrect.


Double clicking single character words ("1", "i", "n", etc.) doesn't select them in some circumstances

Tested versions

Reproducible in: 4.4.dev2, 4.3.stable, and 4.0.4.stable
Not Reproducible in 3.5.3.stable, 3.0.6.stable, and other Godot 3.0 builds

System information
Godot v4.3.stable (77dcf97d8) - NixOS #1-NixOS SMP PREEMPT_DYNAMIC Wed Sep 18 17:24:10 UTC 2024 - X11 - GLES3 (Compatibility)
Issue description
Double clicking single character identifiers/words does not automatically select them, similar to how double clicking parenthesis or braces does not select them.
Steps to reproduce

Place this code in any script in the editor:

func print_ten():
	for i in range(10):
		print(i)
	return 1

Double click on the "i" variable and the "1", and also to the left and right of them
Notice that sometimes the word is selected, other times it isn't

Minimal reproduction project (MRP)
Not necessary - just follow the steps to reproduce.

duplicate of https://github.com/godotengine/godot/issues/89551


removed grunt from ci

[Current coverage][cc-pull] is 97.39%

Merging [#9][cc-pull] into [github-production][cc-base-branch] will increase coverage by +7.42%

2 files (not in diff) in td/public/app were created. more
3 files (not in diff) in td/controllers were deleted. more
6 files (not in diff) in td/config were deleted. more
4 files (not in diff) in td were deleted. more
2 files (not in diff) in .../public/app/services were modified. more

Misses -28
Hits -7

6 files (not in diff) in td/public/app were modified. more

Misses -6
Hits -25

@@           github-production         #9   diff @@
===================================================
  Files                     34         23    -11
  Lines                   1376       1035   -341
  Methods                   58          0    -58
  Messages                   0          0
  Branches                  23          0    -23
===================================================
- Hits                    1238       1008   -230
+ Misses                   138         27   -111
  Partials                   0          0

Powered by Codecov. Last updated by [6074ec8...6b3c3b1][cc-compare]
[cc-base-branch]: https://codecov.io/gh/mike-goodwin/owasp-threat-dragon/branch/github-production?src=pr
[cc-compare]: https://codecov.io/gh/mike-goodwin/owasp-threat-dragon/compare/6074ec802bd32d21fd89647cec9e98a69293b984...6b3c3b163e5dd5ee142c1a191790338ec0784ef2
[cc-pull]: https://codecov.io/gh/mike-goodwin/owasp-threat-dragon/pull/9?src=pr


[Alert] Error MUI: Unsupported `` color. The following formats are supported: #nnn, #nnnnnn, rgb(), rgba(), hsl(), hsla(), color().

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Current behavior 😯
I have implemented a Light and Dark palette and I wanted to use different colors for the default variant of the Alert component, on light mode, everything works fine but as soon as I toggle dark mode my application crashes with this error:
Error
MUI: Unsupported `` color.
The following formats are supported: #nnn, #nnnnnn, rgb(), rgba(), hsl(), hsla(), color().

Oddly enough, the only property that seems to be working correctly is severity warning.
I checked my style overrides but they seem to be pointing to the correct colors, I even tried using the exact same colors in both palettes and the issue still happens.
Expected behavior 🤔
I expected to see the Alert with the colors specified in my dark palette and not an error.
Steps to reproduce 🕹
I managed to replicate the error I am getting on my project using the demo project.
https://codesandbox.io/s/filledalerts-material-demo-forked-y4el8?file=/demo.tsx
Context 🔦
I wanted to implement a dialog with an info alert to provide feedback for my users, my application supports light and dark mode. As soon as I switched to dark mode everything crashed.

  Output from `npx @mui/envinfo` goes here.


improvement: OSIS-50-update-tag-for-release-v0.4.1

Updates tag to OSIS v0.4.1 contains fix for CVE-2019-8457

Waiting for approval
The following approvals are needed before I can proceed with the merge:

the author

2 peers

Codecov Report

Merging #81 (36de5c7) into development/1.0 (f0fdb74) will not change coverage.
The diff coverage is n/a.

@@                Coverage Diff                 @@
##             development/1.0      #81   +/-   ##
==================================================
  Coverage              84.84%   84.84%
  Complexity               267      267
==================================================
  Files                     19       19
  Lines                   1287     1287
  Branches                 107      107
==================================================
  Hits                    1092     1092
  Misses                   135      135
  Partials                  60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0fdb74...36de5c7. Read the comment docs.

/approve

In the queue
The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.
The changeset will be merged in:

:heavy_check_mark: development/1.0

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.
IMPORTANT
Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.
The following options are set: approve

I have successfully merged the changeset of this pull request
into targetted development branches:

:heavy_check_mark: development/1.0

Please check the status of the associated issue OSIS-50.
Goodbye anurag4dsb.


Server and X-Powered-By headers

I had an internal security review on our app today, and the audit pointed out that the server identify itself in http headers:
Server: Kestrel
X-Powered-By: ASP.NET

I think that it's usually accepted that identifying the server software could lead to easier / automated exploits of known security breaches. As such, publicly disclosing it is not a best practice.
Is it a good idea to identify Kestrel by default on a new website? What's the benefit of doing so?
And how do I remove those two headers? I knew where they were in IIS, but I wasn't expecting them from Kestrel.

See https://github.com/aspnet/Hosting/issues/571

ah wrong repo it seems. Thanks, shall I close this ticket?