ClassCastException while accessing RSSI

Hi @xrucka, I finally found some time to try to "productionize" your great effort to bring a proper support for DBus based adapters.
Good news - it is working! Bloody good news! You are rock!
Now, there are few issues we need to resolve. I'll be adding them one by one. Here is the first one.
Caused by: java.lang.ClassCastException: java.lang.Short cannot be cast to org.freedesktop.dbus.Variant
	at cz.organovabanka.bluetooth.manager.transport.dbus.PropertyCache.get(PropertyCache.java:59) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezDevice.getRSSI(BluezDevice.java:538) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezAdapter.lambda$getDevices$13(BluezAdapter.java:262) ~[?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) ~[?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:?]
	at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2746) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:?]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezAdapter.getDevices(BluezAdapter.java:263) ~[?:?]

I do nothing special... Here is another one:
15:03:04.158 [WARN ] [oth.manager.impl.BluetoothManagerImpl] - Device discovery job error
java.lang.ClassCastException: java.lang.String cannot be cast to org.freedesktop.dbus.Variant
	at cz.organovabanka.bluetooth.manager.transport.dbus.PropertyCache.get(PropertyCache.java:59) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezDevice.getName(BluezDevice.java:504) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezFactory.convert(BluezFactory.java:389) ~[?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezFactory.lambda$getDiscoveredDevices$8(BluezFactory.java:297) ~[?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) [?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) [?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) [?:?]
	at java.util.concurrent.ConcurrentHashMap$ValueSpliterator.forEachRemaining(ConcurrentHashMap.java:3566) [?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) [?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) [?:?]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) [?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:?]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) [?:?]
	at cz.organovabanka.bluetooth.manager.transport.dbus.BluezFactory.getDiscoveredDevices(BluezFactory.java:298) [219:organovabanka.org.eclipse.smarthome.binding.bluetooth.transport.dbus:1.0.0.SNAPSHOT]
	at org.sputnikdev.bluetooth.manager.impl.BluetoothManagerImpl$DeviceDiscoveryJob.discoverDevices(BluetoothManagerImpl.java:690) [214:org.sputnikdev.org.eclipse.smarthome.binding.bluetooth:1.1.5]
	at org.sputnikdev.bluetooth.manager.impl.BluetoothManagerImpl$DeviceDiscoveryJob.run(BluetoothManagerImpl.java:683) [214:org.sputnikdev.org.eclipse.smarthome.binding.bluetooth:1.1.5]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
	at java.lang.Thread.run(Thread.java:748) [?:?]

I'm going to try your fix. I'll let you know.

Sorry, I was completely out of action with my sickness. I've been monitoring it with your fix and never saw that exception so far. I guess you've fixed it. Cheers.
However, there are a few other issues I faced. I'l be adding them shortly.

This one is a lot more serious. It - at least in theory - should not occur unless either DBus or Bluez dies. Or perhaps, if there was different dbus-based extension closing the dbus connection (which, however, should not occur). How long were you waiting before restarting the openhab?

Hi @xrucka, are you referring #6 ? If so, I could not catch that moment when it starts doing so. I will keep monitoring it.

Morning, @vkolotov yep, I somehow managed to reply into the wrong window and not notice it :-/

(no reference in commit, just numbering :-/)


Add support for macOS

Can't use apk to install packages on macOS, so we need another option.
macOS comes with git, python3-pip, and python3-venv by default but not cmake.
Need to either download the Cmake.app package (dmg or tarball) directly from Cmake or install brew+cmake.
The first option probably makes more sense since there's less overhead.

Permissions for the Terminal to communicate with USB devices requires the user Allow permission manually by clicking a popup.

Made progress on the add-macos-support branch. Almost ready to merge.
Still to-do:

Install cmake automatically
Detect device automatically

Need to try installing cmake into
/Users/${USER}/Applications/

Found an option for detecting the USB serial number without throwing an error when not detected.


Add a color to a snake

Def of done : It is done when each snake have a different colour.

👍

Fixed in #62


privledged vs. privileged

The nmap privileged option does not work due to was appears to be a typo of privledged vs. privileged.

Fixed by #32.


[Feature]: "Keep app open" reminder when viewing LN invoice

Describe the problem
Bitkit needs to remain in the foreground to receive payment via LN. We know some users will produce an invoice and copy it to send from another app on the same device, this will often not work.
See internal discussion https://synonymworkspace.slack.com/archives/C01H106JRQB/p1723735706112959
Describe the solution
We could do with a reminder on the LN invoice page so the user is aware of this limitation.
Additional context
Include in automated test suite.

The user should have a short time window available to pay themselves using with a different Lightning wallet, even when switching between apps, where Bitkit is pushed to the background. We have tested this ourselves and have seen users do this successfully in BTC Prague. Perhaps this does not work for CJIT invoices somehow? That would be odd, as these are Blocktank invoices afaik and thus don't require the user to be online.

The window where it works is very short I think. If we are going to make decisions based on this, we should test the limits of how long this works and be sure it is acceptable. My guess is that the window is not long enough to accommodate most real-world situations, but I am just guessing.

@catch-21 could you perhaps test what the available time window is on iOS & Android to pay between Lightning wallets?

I can try but it may vary by device. I've never had an issue using two apps this way but my device has plenty of RAM and it's up to the OS to manage resources. Modern versions of Android and iOS aim to allow apps to complete background tasks for a few minutes after being closed. However, older devices with resource-intensive apps will more likely experience the OS reallocating resources sooner as a priority. You also have the option to be more aggressive with battery optimisation.

added to prototype v52


[Bug]:  When ever i try to deploy any application i get error of compose is not a docker command

Error Message and Logs

i am using raspberry pi 4b
Steps to Reproduce

Example Repository URL
No response
Coolify Version
v4.0.0-beta.379
Are you using Coolify Cloud?
No (self-hosted)
Operating System and Version (self-hosted)
No response
Additional Information
No response

Hey, this could be an issue if your docker version is outdated.
Can you please show me the output of "docker --version"?


[template] Position of template object resets to 0

I'm having some issues when applying the following template:
<script id="boxesTemplate" type="text/html">
            <a-box width="0.2" height="1" depth="0.2" position="0 0.5 -0.2"></a-box>
            <a-box width="0.2" height="1" depth="0.2" position="-0.2 0.5 0.2"></a-box>
            <a-box width="0.2" height="1" depth="0.2" position="0.2 0.5 0"></a-box>
            <a-box width="1" height="0.2" depth="1"></a-box>
</script>

When applied to an entity the child <a-box> entities come through, but their position resets to 0:
<a-entity template="src: #boxesTemplate;"></a-entity>

I'm using aframe-template-component v3.2.1 from npm

Seems like moving the above template into my HTML so it isn't included during the compile step fixes this issue. I'm using this component with ember.js and ember-aframe: https://github.com/ember-vr/ember-aframe

same problem here using react and react-dom

ok a "normal"  project works fine
glitch.com/edit/#!/kind-scourge

this sample example with react does not work
`/* global  AFRAME */
import AFRAME from 'aframe'
import 'aframe-template-component'
import React, { Component } from 'react';
import logo from './logo.svg';
import './App.css';
class App extends Component {
render() {
return (
  <a-scene>
  <a-assets>
   <script id="clouds" type="text/html">
    <a-text class="nametag" value="CLOUDS!" side="double" position="0 2 0" rotation="0 0 0" color="black"></a-text>
    </script>
  </a-assets>

  <a-entity id="light" light="type: ambient; color: #888"></a-entity>
   <a-entity template="src: #clouds"></a-entity>
</a-scene>
);

}
}
export default App;`

@ngokevin here is a test react project
yarn install
yarn start
templatebug.zip

I'll try to look some time. Feel free to debug the component, it's not too complicated.


Chrome Autofill: the label is not floated

What Angular MDC version are you using?
0.43.3
What browser(s) is this bug affecting?
Chrome  71.0.3578.98, maybe more browsers

@trimox try to look at this: https://github.com/material-components/material-components-web/issues/4447#issuecomment-467214641

@ofekashery Thanks for the heads up. Taking a look.

Duplicate of https://github.com/trimox/angular-mdc-web/issues/1813


Refresh page when switching networks

When user changes networks, refresh the page to ensure keys are reset.
Needed because #352 hasn't resolved wrong key for network issues.

Closed by #372


Okta integration does not have SSO Outputs

It would be great if the okta integration had outputs that mapped to the URLs and information available in the UI.  This could then be consumed by other TF to make use of the integration directly without having to copy and paste from the UI.
https://github.com/cyralinc/terraform-provider-cyral/blob/main/doc/resource_integration_okta.md

@greenkiwi , have you seen this module? https://github.com/cyralinc/terraform-cyral-idp-okta/
Is this what you are looking for?

@wcmjunior Yes and no.  I think that having the module is awesome.
That being said, I think that the endpoints should be Attributes Reference on the cyral_okta resource rather than having to pull them from data and/or generate them via special knowledge in locals.   This might mean that the TF mapping to the API needs to create the object and then poll to fetch the information once the object has been fully created, assuming something happens asynchronously.

Similarly, in that module - I think that you don't need to use the data.okta_app_metadata_saml as it's all available from the okta_app_saml attribute reference

Closing this ticket as we are deprecating the vendor-specific integrations and focusing only on the generic SAML from terraform-provider-cyral:v4.2.0 and CP v4.6.0 onwards.


Document EntityId logging behaviour in the cloud.

Description
Added a sentence to the LoggingUtils.EntityId variable documentation, to document why it might be useful to provide. This relates to the imminent release of entity id filtering in the Inspector and Logger.
Tests
No tests.
Documentation
See above.
Primary reviewers
@jessicafalk

closing this in favor of https://github.com/spatialos/UnityGDK/pull/427


Update additional Archetype Journal Entries to use @Embed

All the undead archetypes

@simonward - Both suggestions have been committed. For some reason, GitHub didn't like me applying the suggestions so I had to go into the file and edit it for them to stick—no clue what's going there


Microsoft Azure Storage Explorer doesn't see any documents in container

Platform/OS: Windows 10
Architecture: x64
Bug Description
Storage Explorer doesn't see any documents in any containers

On portal

Steps to Reproduce

Open Explorer
Select CosmosDb
Select database
Click on Documents

Expected Experience
Display data like in the portal
Actual Experience
No data in containers

I found that it doesn't show any documents when I don't have Discriminator for data. In EF:
builder.HasNoDiscriminator

Are there any plans to fix that?

Are there any plans to fix that?

Hi @southpolesteve are there any plans to fix that?

The Cosmos team recently deprecated our integration with Azure Storage Explorer. Here is the full deprecation notice from the 1.15.1 Storage Explorer release notes:

The Azure Cosmos DB integration with Storage Explorer has been deprecated. Any existing functionality will not be removed for a minimum of one year from this release. Users should use the Azure Portal, the Azure Portal desktop app, or the standalone Cosmos Explorer instead – all of which contain many new features that aren’t currently supported in Storage Explorer. Additionally, the Azure Cosmos DB team has also open sourced the data explorer project and will be investing in their own desktop tooling at a later date. Please keep an eye on their repository for more information.

Currently, we are not prioritizing new features or bug fixes for Cosmos in Azure Storage Explorer. If you the issue you are reporting also applies to the Azure Portal or https://cosmos.azure.com, please file an issue in our open source Cosmos Explorer repo


Add Docker and update CI

Add Docker and update CI

No drastic changes other than Ci and the Dockerfile. Travis passes: https://travis-ci.org/AnsibleShipyard/ansible-zookeeper/builds/53513941
(wonder why it doesn't chime in here)
Going to merge.


E-book links

Here more details on how to handle the e-books:
https://its.cern.ch/jira/browse/CDS-1126
( refers to https://github.com/CERNDocumentServer/cds-dojson/issues/178)
Here a relevant SNOW ticket about EZProxy:

https://cern.service-now.com/nav_to.do?uri=u_request_fulfillment.do?sys_id=96b5b6a04f6e4740b9cc09de0310c76a%26sysparm_view=tabs_new
http://library.cern/resources/remote

Documentation from EbookCentral: https://support.proquest.com/articledetail?id=kA11W000000XbrxSAC
( requires https://github.com/CERNDocumentServer/cds-dojson/issues/201 )

Implemented in https://github.com/CERNDocumentServer/cds-ils/issues/31


Add a check for as many of the security best practices outlined by Zeke here

where are the best practices outlined?


[SPARK-41591][PYTHON][ML][FOLLOW-UP] Fix type hints that are incompatible with Python <= 3.8

What changes were proposed in this pull request?

This PR fixes type hints (added in https://github.com/apache/spark/pull/39188) that are incompatible with Python <= 3.8.
Why are the changes needed?

Because PySpark still supports Python <= 3.8.
Does this PR introduce any user-facing change?

No
How was this patch tested?

Existing unit tests

Merged to master.

Thank you for catching this!


$AURDEST directory becomes cluttered

The $AURDEST directory tends to become cluttered over time, mostly with symlink and leftover of the makepkg build process, even though the sources and built packages are configured to be stored elsewhere (SRCDEST, PKGDEST).
Find a way to keep it clean, maybe by extending the -Sc operation. Keep in mind that extra files are also kept there if some of the env variables ($SRCDEST, $PKGDEST, $LOGDEST, ..) aren't set in makpekg.conf.

I might be missing something, but doesn't AURDEST only contain the git
clone directories, where the actual build is performed?
On 31.03.2016 14:56, Remy Marquis wrote:

The |$AURDEST| directory tends to become cluttered over time, mostly
with symlink and leftover of the makepkg build process, even so the
sources and built packages are configured to be stored elsewhere
(SRCDEST, PKGDEST).
Find a way to keep it clean, maybe by extending the -Sc operation.
Keep in mind that extra files are also kept there if some of the env
variables ($SRCDEST, $PKGDEST, $LOGDEST, ..) aren't set in makpekg.conf.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/rmarquis/pacaur/issues/461

doesn't AURDEST only contain the git clone directories, where the actual build is performed?

Yes. Isn't that exactly what I wrote above?

Oh, you mean cluttering of the /sub-directories/ of AURDEST. :D
In the absence of LOGDEST, you could create a "log" directory in a clone
directory, and pass the LOGDEST variable directly to makepkg, e.g
mkdir -p log
LOGDEST=${LOGDEST:-./log} makepkg
You could remove symbolic links with find, though this is hairy:
[[ -d $AURDEST ]] && find "$AURDEST"/* -maxdepth 1 -type -l -delete
Otherwise I'd let makepkg -c and friends do the job.
On 31.03.2016 18:57, Remy Marquis wrote:

doesn't AURDEST only contain the git clone directories, where the
actual build is performed?

Yes. Isn't that exactly what I wrote above?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/rmarquis/pacaur/issues/461#issuecomment-204020384

Yeah, subdirectories, my bad. :)
The makepkg -c option is already passed by default, I'll have to check why some leftover files aren't removed. Might actually be because PKGDEST is defined - the symlink is still created.

The makepkg -c option indeed doesn't remove symlink to build packages and log files.
In pacaur, log files are only created when the --silent option is set.

FWIW, I use this script to clean up AURDEST:
#!/bin/sh
readonly argv0=cleanup.sh
cd /home/aur || exit

# Remove regular files in the parent directory.
find -maxdepth 1 -type f -not -name "$argv0" -delete

# Remove source tarballs.
find -maxdepth 2 -type f -name '*.tar.gz' -delete

# Remove backup files.
find -maxdepth 2 -type f -name '*~' -delete

# Remove symbolic links to packages.
find -maxdepth 2 -type l -name '*.pkg*' -delete

# Remove non-empty pkg/src directories.
find -maxdepth 2 -type d -name pkg -or -name src -print0 | xargs -0 rm -r

Thanks!

Note that all of the above can be replaced with git clean:
find -maxdepth 2 -type d -name '.git' -execdir git clean -xi \;

This includes a nice selection dialog for what files you want to delete.

Thanks, git clean is indeed cleaner to use here, although I'd prefer to clean automatically without the interactive part.
Also, I probably should think about some option to remove clone directories of packages that aren't installed anymore.

From #559:

Maybe git clean should be run automatically after a successful build, and also extend -Scc to delete the git folders of uninstalled packages as two separate features?

I am unsure about automatic cleaning after a successful build. This isn't what makepkg does, and users that haven't configured PKGDEST or SRCDEST expect the source and build files to be there.

Yes it should not be automatic and at least let the user decide what to do if added.

Done in 10baa520f66b1fabd7eed7c6596bc74cb1a3940b, as a -Sc command extension.


267 populate homepage

Description

Could still use a hand making it prettier Im sure
Also most of the articles pulled had no images. Maybe I am getting these incorrectly maybe XDD data is not predictable
Lastly this needs to be corrected. I cannot create a commit with an await in a for loop. Im sure there is a better way to do this but i am not aware of it

https://user-images.githubusercontent.com/17088680/205547604-5d91e24e-7f52-4891-b2c3-a51db48a2e48.mov
Resolves #(issue)
#267
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Currently only manually.
I will need to write a fair amount of test cases going forward but i figured this could come as a second PR to let Pascale play around with these changes
Please describe the tests that you ran to verify your changes. Provide reproduction instructions and list any relevant details for your test configuration
Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

@YohannParis
I have tried filtering only articles that have images. This is currently a slow process as for each document I need to pull its artifacts and check if it has an image.
I just ran a test to see if I could find any with the search "Covid 19" pulling 10,000 documents.
Of all of these documents only 800 had an abstract and of those 800 none of them had images.

I could store a bunch of mock images and randomly assign these to articles instead of the blank image i currently have, or we can leave them as blank. Thoughts?


aspnet/core/client-side/angular "Using AngularJS for Single Page Applications (SPAs) with ASP.NET Core"

Review / update

@Rick-Anderson Is this a duplicate of https://github.com/aspnet/Docs/issues/3543?

@scottaddie  yup


visa_handle.clear on instantiation causes some instruments to freeze

The use of visa_handle.clear on the instantiation of a VISA instrument causes some instruments to lock up. In particular I've noticed for SERIAL instruments, this command spews some garbage on the serial port that requires the instrument to be reset before it is possible to use it again.
Steps to reproduce

Plug in a buggy instrument and instantiate an instance of it.

Actual behaviour
The instrument freezes
System
Win 7
Using the Harvard DecaDAC
2b6d72b

@WilliamHPNielsen ever noticed the same ?

To expand on this issue, some versions of the VISA library automatically send *CLS; on receiving a viClear. This sequence seems to crash older versions of the decaDac. See: http://digital.ni.com/public.nsf/allkb/4A8D61E759174E7F86256E20005A8ABC
A couple of options we could choose between:

Don't viClear when opening an instrument. This is probably the simplest option, my preferred.
If we are worried about junk in the IO buffers, use viFlush instead:

self.visa_handle.flush(visa.constants.VI_READ_BUF_DISCARD | visa.constants.VI_WRITE_BUF_DISCARD)

Set the VISA flags given in the linked document such that we don't assume that VISA instruments support SCPI commands.

@spauka I am not sure I understand the issue, or I guess there's two part:

the ni-visa library  (before version 3) sends automatically a *CLS; which some instruments don't support
wether or not we want to have qcodes to send.clear at the beginning (whether or not this is a NoOp I couln't find the visa documentation )

@giulioungaretti For each of your two points above:

Yes, this locks up our version of the harvard dac on certain computers
visa_handle.clear empties the IO buffers for instruments, so it isn't quite a no-op. Hence I think we do actually want to send this for instruments that support it.

I committed a new change that maintains the existing behavior of clearing the IO buffers for other instrument types, but uses viFlush for serial ports, which is a specific method for clearing buffers that only seems to exist for Serial instruments.


SMTP notification not working with german web UI

Hi,
i came accross with an bug using OPNsense with the web UI set to german language (https:///system_general.php).
When i try to validate the SMTP notification configuration with web UI language set to english, everything is workin fine. If i change the language to german, nothing happens (click on "Test SMTP" @ https:///system_advanced_notifications.php). I do not receive an E-Mail with web UI language set to german.
I tested it with versions: 17.7.10, 17.7.11
Currently running:
OPNsense 17.7.11-amd64, FreeBSD 11.0-RELEASE-p17
Best regards
Wolfgang Hofbauer

Hi Wolfgang,
Thanks, I'll take a look.
Cheers,
Franco

Hi Franco,
i received the missing test e-mails in my spam folder. All with english web UI in my Inbox, all with german web UI in spam folder.
Hope this helps.
Wolfgang

Sounds like a spam filter issue on the receiving end then, not a bug. What are you testing against? Maybe a German free mail provider?

I'm hosting an email server with postfix, dovecot and rspamd.
As you can see in the attached screenshot, the email with german language has an much higher score rating (second score result):


Papd 18 change external references to eit

Similar to the Client changes, I've bumped the version in use here to signify a breaking change in the bucket reference links and to break from the current versioning.
I've also incorporated the changes we're currently using in Hostile-S3 so this would then supersede that repo.
Links to this ticket

Additional testing

Confirmed hostile fetch can pull the index files to a new hostile-eit application folder. Users can copy across their local files instead of re-downloading if they wish.

Installed in a local Pathogena Client Conda environment and ran hostile on fastq files

16:15:19 INFO: EIT Pathogena client version 2.0.0rc1
16:15:19 INFO: Performing FastQ checks and gathering total reads
16:15:19 INFO: Calculating read count in: /Users/jdhillon/code/pathogena/client/tests/data/reads/tuberculosis_1_1.fastq
16:15:19 INFO: Calculating read count in: /Users/jdhillon/code/pathogena/client/tests/data/reads/tuberculosis_1_2.fastq
16:15:19 INFO: 2.0 reads in FASTQ file
16:15:19 INFO: Removing human reads from ILLUMINA FastQ files and storing in /Users/jdhillon/code/pathogena/client
16:15:19 INFO: Hostile version 2.0.0rc1. Mode: paired short read (Bowtie2)
16:15:19 INFO: Fetching standard index human-t2t-hla-argos985-mycob140 (https://objectstorage.uk-london-1.oraclecloud.com/n/lr3yhdniv6gu/b/human-genome-indices/o/human-t2t-hla-argos985-mycob140.tar)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.24G/4.24G [12:55<00:00, 5.87MB/s]
16:28:15 INFO: Verifying checksum be0d63ed39d9eeb9657e936c3ded7661c1ef64d8a0de3c1acaf92348c9c8993d…
16:28:25 INFO: Extracting human-t2t-hla-argos985-mycob140.tar…
16:28:28 INFO: Downloaded standard index /Users/jdhillon/Library/Application Support/hostile-eit/human-t2t-hla-argos985-mycob140
16:28:28 INFO: Cleaning…
16:28:29 INFO: Cleaning complete
16:28:29 INFO: Human reads removed from input samples and can be found here: /Users/jdhillon/code/pathogena/client

Tested pathogena upload with an Illumina sample with human read removal|decontamination enabled (sample)

16:48:01 INFO: EIT Pathogena client version 2.0.0rc1
16:48:02 INFO: Removing human reads from ILLUMINA FastQ files and storing in /Users/jdhillon/code/pathogena/client
16:48:02 INFO: Hostile version 2.0.0rc1. Mode: paired short read (Bowtie2)
16:48:02 INFO: Found cached standard index human-t2t-hla-argos985-mycob140
16:48:02 INFO: Cleaning…
16:48:29 INFO: Cleaning complete
16:48:29 INFO: Human reads removed from input samples and can be found here: /Users/jdhillon/code/pathogena/client
16:48:31 INFO: Uploading sample1-jay
16:51:43 INFO:   Uploaded ab40758e-5161-46c7-b0ec-8c054a29c218_1.fastq.gz
16:51:43 INFO: Uploading sample1-jay
16:55:09 INFO:   Uploaded ab40758e-5161-46c7-b0ec-8c054a29c218_2.fastq.gz
16:55:09 INFO: Upload complete. Created uzyw9b.mapping.csv (keep this safe)

And could you please double check the following things:

We have a secret OCI_USERNAME_GPASLTD
We have a secret OCI_TOKEN_GPASLTD
The username & token can be used to push to gpasltd
There is already a hostile-eit inside gpasltd, we must create this prior to the push, otherwise it will result in some errors

And could you please double check the following things:

We have a secret OCI_USERNAME_GPASLTD
We have a secret OCI_TOKEN_GPASLTD
The username & token can be used to push to gpasltd
There is already a hostile-eit inside gpasltd, we must create this prior to the push, otherwise it will result in some errors

I can confirm these exist and I've tested logging into the registry with these credentials:

I've created hostile-eit inside OCIR


Adding BERT

I added BERT (english version) - results for Reddit are very bad?

LGTM after rename to BERTEncoder, though I am still surprised about the results


Report from the Sufia UI Working Group

Recap of the group's recommendations for the upcoming Sufia 7–thoughts on the diverse needs of the community and the result, what worked, what didn't work, how to approach it next time.
Presentation
Managers, Developers, UX, Stakeholders
Michael Tribone +

8/20  Hello Richard,
Thanks so much!

Absolutely
At present no co-presenters, but am asking members of the UI group.
Yes
Wiil do
Registered

Thanks again,
Michael


Fix End to End tests

Task/Issue URL: https://app.asana.com/0/1206329551987282/1208028109665424/f
Description:
I’ve opened a draft PR that implements some of the recommendations highlighted in this comment to have some feedback on the approach.
In particular this PR removes the onboarding_browsing.yaml file and the need to run this nested flow from every workflow to handle the dax dialogs.
This has been replaced by using launch arguments and environment variables in setup.yaml
// In setup.yaml
arguments:
        isUITesting: true
        isOnboardingCompleted: ${ONBOARDING_COMPLETED}

With the above we can pass values from Maestro to our App and set the App in a particular state:

isUITesting is used in the App to check if UI tests are running.
ONBOARDING_COMPLETED is defined as an environment variable that we can pass via command line when we run the tests. e.g.

maestro cloud --apiKey ${{ secrets.MAESTRO_CLOUD_API_KEY }} -e ONBOARDING_COMPLETED=true --fail-on-timeout=true --fail-on-cancellation=true --timeout=150 --ios-version=17 --include-tags=release DerivedData/Build/Products/Debug-iphonesimulator/DuckDuckGo.app .maestro/
In this way we can easily default the existing tests not to show the onboarding passing ONBOARDING_COMPLETED=true while maintaining the flexibility to pass false for tests related to the onboarding.
We could use this approach also for experiment variant e.g.
arguments:
        …
        onboardingVariant: ${ONBOARDING_VARIANT}

And pass the variant value via the command line.
Next Steps / Future improvements:

Show / hide onboarding intro in UITests based on ONBOARDING_COMPLETED and remove shared/onboarding.yaml
Improve end-to-end.yaml workflow

Steps to test this PR:

The latest test run has all the tests passing apart from one that timed out (06_mjs_heuristic_ad_domain_provided_but_empty_dsl_not_needed (20m 56s). This though is because of Maestro not because of the approach. When I re-run manually the test in Maestro cloud the test passed

Run maestro locally

Definition of Done (Internal Only):

[ ] Does this PR satisfy our Definition of Done?

Copy Testing:

[ ] Use of correct apostrophes in new copy, ie ’ rather than '

Orientation Testing:

[ ] Portrait
[ ] Landscape

Device Testing:

[ ] iPhone SE (1st Gen)
[ ] iPhone 8
[ ] iPhone X
[ ] iPhone 14 Pro
[ ] iPad

OS Testing:

[ ] iOS 15
[ ] iOS 16
[ ] iOS 17

Theme Testing:

[ ] Light theme
[ ] Dark theme

Internal references:
Software Engineering Expectations
Technical Design Template

@alessandroboron I like this. I agree we can reuse this approach also for other things. And thing change should make UI tests faster as they won't have to "solve" onboarding on every single test.

@loremattei @alessandroboron
My worry is that something we do affects onboarding or visa versa and we'd never spot that.  Onboarding in particular is quite hard for us to see breaking changes just because it's quite rare for us to run the app from deleted state in production.  However if you're OK with that risk I don't have objections.

@brindy yep, that's a good point.
I think that using @alessandroboron's approach we can add tests specifically focused on onboarding. E.g. we could add a new test target and invoke it with a separate GHA command that passes ONBOARDING_COMPLETED=false and the variant we want to test (Alessandro mentioned in the task description that this is doable with this same approach). As a starting point, we could test the current onboarding only.

@brindy yep, that's a good point.
I think that using @alessandroboron's approach we can add tests specifically focused on onboarding. E.g. we could add a new test target and invoke it with a separate GHA command that passes ONBOARDING_COMPLETED=false and the variant we want to test (Alessandro mentioned in the task description that this is doable with this same approach). As a starting point, we could test the current onboarding only.

@loremattei , @brindy correct the intention was to also write explicit tests for the onboarding. I’ve actually updated the PR and used the same approach to inject the variant in UI Tests.
You can test it by checking out the code and run:

./setup_ui_tests.sh to setup the simulator for maestro.
./run_ui_tests.sh onboarding_tests to run onboarding tests.

About onboarding_tests
01_control_group_onboarding.yaml -> Tests the control group linear flow.
02_control_group_hide_onboarding.yaml -> Tests the control group hide dax dialogs.
03_experiment_group_linear_onboarding.yaml -> Tests the experiment linear flow.
Disclaimer: The tests may not be perfect, e.g. find UIElements by strings rather than accessibility identifiers. I tried to re-use what was already there to show how we can do that.
Since I think this PR will fix the tests failing on main I will convert the draft to a PR.


Update spring-boot-starter-test, ... to 2.4.6

Updates

org.springframework.boot:spring-boot-starter-test
org.springframework.boot:spring-boot-starter-web

from 2.4.5 to 2.4.6.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (2.4.5).
You might want to review and update them manually.
notes/0.56.0.md

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.springframework.boot" } ]

labels: library-update, semver-patch, old-version-remains

Superseded by #1135.


Master not created for pipeline output

My setup is:

kubernetes 1.5.6
pachyderm 1.4.3
deployed in GKE

When I run this pipeline (image is public and available for reproducing this):
{
  "pipeline": {
    "name": "model"
  },
  "transform": {
    "image": "combustml/pmd-training:0.1-SNAPSHOT",
    "cmd": [
	"pmd-training",
	"airbnb",
	"-t",
	"random-forest",
	"-i",
	"/pfs/training/airbnb.clean.avro",
	"-o",
	"/pfs/out/model.zip",
	"-s",
	"/pfs/out/summary.txt",
	"-J-Xmx2048m"
    ]
  },
  "inputs": [
    {
      "repo": {
        "name": "training"
      },
      "glob": "/"
    }
  ]
}

I get a failed job:
➔ pachctl list-job
ID                                   OUTPUT COMMIT STARTED       DURATION  RESTART PROGRESS STATE
29606b89-b4c4-4156-8474-72c805f8681e model/-       6 seconds ago 1 seconds 0       0 / 1    failure

It creates the model repo as expected:
➔ pachctl list-repo
NAME                CREATED             SIZE
model               8 minutes ago       0 B
training            11 minutes ago      7.015 MiB

But, when I then try to fix and update the pipeline, I get:
➔ vim pipeline.json
➔ pachctl update-pipeline -f pipeline.json
pachyderm_pfs/commits/model/ master not found

Moreover, the update actually triggers a job that also fails.  However, when I try to get the logs for that file, there is nothing.  Not even raw logs:
➔ pachctl list-job
ID                                   OUTPUT COMMIT STARTED            DURATION  RESTART PROGRESS STATE
a22a17a1-f33e-47bc-80a9-e8b7c7764ea8 model/-       About a minute ago 2 seconds 0       0 / 1    failure
29606b89-b4c4-4156-8474-72c805f8681e model/-       2 minutes ago      1 seconds 0       0 / 1    failure
➔ pachctl get-logs --job a22a17a1-f33e-47bc-80a9-e8b7c7764ea8 --raw
{"pipelineName":"model","pipelineId":"1e84ebbc771743f6b3955744853711e5","jobId":"a22a17a1-f33e-47bc-80a9-e8b7c7764ea8","workerId":"pipeline-model-v2-ltjhw","data":[{"hash":"+C0vmqIamWW33Z8mYHA3ppVMloBxfr9XFFIrv7leesU="}],"ts":"2017-04-10T17:41:37.768809064Z","message":"Received request"}
{"pipelineName":"model","pipelineId":"1e84ebbc771743f6b3955744853711e5","jobId":"a22a17a1-f33e-47bc-80a9-e8b7c7764ea8","workerId":"pipeline-model-v2-ltjhw","data":[{"hash":"+C0vmqIamWW33Z8mYHA3ppVMloBxfr9XFFIrv7leesU="}],"ts":"2017-04-10T17:41:38.041590272Z","message":"input has not been processed, downloading data"}
{"pipelineName":"model","pipelineId":"1e84ebbc771743f6b3955744853711e5","jobId":"a22a17a1-f33e-47bc-80a9-e8b7c7764ea8","workerId":"pipeline-model-v2-ltjhw","data":[{"hash":"+C0vmqIamWW33Z8mYHA3ppVMloBxfr9XFFIrv7leesU="}],"ts":"2017-04-10T17:41:38.157569837Z","message":"beginning to process user input"}
{"pipelineName":"model","pipelineId":"1e84ebbc771743f6b3955744853711e5","jobId":"a22a17a1-f33e-47bc-80a9-e8b7c7764ea8","workerId":"pipeline-model-v2-ltjhw","data":[{"hash":"+C0vmqIamWW33Z8mYHA3ppVMloBxfr9XFFIrv7leesU="}],"ts":"2017-04-10T17:41:38.157921836Z","message":"running user code"}
{"pipelineName":"model","pipelineId":"1e84ebbc771743f6b3955744853711e5","jobId":"a22a17a1-f33e-47bc-80a9-e8b7c7764ea8","workerId":"pipeline-model-v2-ltjhw","data":[{"hash":"+C0vmqIamWW33Z8mYHA3ppVMloBxfr9XFFIrv7leesU="}],"ts":"2017-04-10T17:41:38.158028033Z","message":"finished processing user input"}
➔ pachctl get-logs --job 29606b89-b4c4-4156-8474-72c805f8681e --raw
➔

Just re-created this with another pipeline.  This seems to always happen after the first run fails. @derekchiang maybe related to the issues we were discussing on Slack?


fix(core): make sure queries that never fetched can be garbage collected

This is important if data comes into the cache via a non-fetching method (like setQueryData or hydration) AND there is no active observer for them. Without this, they would never be gc'ed, but they should be after the default cacheTime
closes #4887

Codecov Report
Base: 91.69% // Head: 91.69% // Increases project coverage by +0.00% :tada:

Coverage data is based on head (50a5311) compared to base (7a62494).
Patch coverage: 100.00% of modified lines in pull request are covered.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4898   +/-   ##
=======================================
  Coverage   91.69%   91.69%
=======================================
  Files         111      111
  Lines        4129     4130    +1
  Branches     1062     1062
=======================================
+ Hits         3786     3787    +1
  Misses        322      322
  Partials       21       21

Impacted Files
Coverage Δ

packages/query-core/src/query.ts
99.02% <100.00%> (+<0.01%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Block access to EC2 Instance Metadata

Currently it's possible to retrieve instance metadata from EC2's link-local metadata IP address. This only grants you the ability to make AWS API calls as the server instance, i.e. have read-only access to S3, but I would still consider it a security problem. I presume it's possible to set up some iptables rules on the host, or something analogous but more user-friendly.

We can probably implement the DOCKER-USER solution suggested in https://ops.tips/blog/blocking-docker-containers-from-ec2-metadata/.

This should be handled by the cgroup setting here https://github.com/raxod502/riju/blob/ff95d0aadb8f7b7af523fdcfd65972842122120e/packer/riju.slice#L14 however, that doesn't work for some reason. Will need to investigate.


Allow disabling splitting original message to lines

Sometimes it's necessary to join multiple messages from the input, without splitting the original message by new lines (example: joining es bulk api input messages)

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then  say 'jenkins, test it'.

Could this be merged?

any update?

@sxmichael Sorry for the very long delays.
I don't understand what problem this code is solving. Multiline is a line-oriented concept, so splitting by line on the input payload makes sense to me (example: tcp input can read large chunks of text that include multiple, possibly unrelated, lines).
Your example of ES bulk api isn't clear enough to me to understand what you need. Can you tell me a story or describe a use case?


Use the editor state passed directly from draftjs

As of Draft 0.10.1 editorState is passed in directly into many handler functions. Unfortunately this changed the arity of these functions and made this plugin incompatible with the latest draft-js and draft-js-plugins: https://github.com/draft-js-plugins/draft-js-plugins/blob/master/draft-js-plugins-editor/CHANGELOG.md#added

Note sure why circleci is failing. It says
Bad response: 422 {"message":"Couldn't find a repository matching this job.","error":true}

cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js returned exit code 1

Thank you for this contribution! I will review few later.

Looks good, thanks!
We will merge later, also publish to npm as version 0.3.0 .

@ngs Would you please merge this?

https://github.com/ngs/draft-js-markdown-shortcuts-plugin/pull/26#issuecomment-305380253

coveralls fails with CI triggered by author without write permission. sorry to make you confused.
anyway, thanks for this PR! LGTM!

Awesome! Thanks guys. Really excellent work with this plugin 👏

@ngs this plugin a great, thank you for building it. And @iansinnott thank you both for getting it working with the latest Draft.js.
@ngs - any timeline on publishing 0.3.0 to npm?
Thanks!

@chasemcaleese Sorry for late action, I've published as 0.3.0. Thanks!

@ngs no, thank you! I really appreciate your work on this plugin 👍


Make calling the user_info endpoint optional

This PR makes it so that the user_info endpoint is not called when the Omiauth setting skip_info is set.
This is useful for example when integrating with Azure AD B2C, which does not expose this endpoint. It may also be useful in cases where we only care about the ID or access token - this will save one HTTP call.
I also run the file through prettifier to make the syntax consistent, I hope that is ok.
Also note the version bump to 0.9.5, but feel free to update this as necessary.

@ondrejpialek
Thank you for your PR.
Could you split it into two PRs, one to format and one to change the function?

@ondrejpialek
Good evening,
Sorry for your extra work, but could you push your patch to the new repository?
Please see the README file.
Thank you.


Clean up form submission handler in test

JSDOM does not allow for a form to be submitted and will throw an error.
To avoid any issues we've prevented it from submitting. This change
relocates that code outside of the button click handler and expands on
the comment to make it clearer they are unrelated and that the submission
handler is an outside necessity.
Changes

Relocated the form submission handler

Testing

Run gulp test:unit

Checklist

[x] PR has an informative and human-readable title
[x] Changes are limited to a single goal (no scope creep)
[x] Code can be automatically merged (no conflicts)
[x] Code follows the standards laid out in the development playbook
[x] Passes all existing automated tests
[x] Any change in functionality is tested
[x] New functions are documented (with a description, list of inputs, and expected output)
[x] Placeholder code is flagged
[x] Visually tested in supported browsers and devices
[x] Project documentation has been updated
[x] Reviewers requested with the Reviewer tool :arrow_right:

looks good, just nix the change to the lock file.

What Steven said

Looks good, just nix the change to the lock file.

Whoops, removed.


XML to HTML tag fix.

The method used to load the spine items into iframes for IE does not (cannot?) maintain the content type. All documents end up being treated as HTML. This breaks EPUBs (in particular the CFI generation) that leverage XML since many self closed tags are invalid in HTML.

The 'fix'/hack for now is to parse the spine item and replace invalid self closing tags with html valid closing tags.

After a slack discussion it was decided to do this for all browsers to maintain consistency.

I did not run tests on this regexp routine, but it looks good to me.

Found a problem with the original regex that broke when a newline was inside the tag. This fixes the issue.


AvaloniaVSCode is stale on OpenVSX at version 0.0.20

Is your feature request related to a problem? Please describe.
Yes, I've had a previewer problem on my leenooks VSCodium, it was stuck at grayed out "Build preview" button!
Describe the solution you'd like
Please upload fresh VSIX files at OpenVSX!
Describe alternatives you've considered
Of course, I've downloaded the files from github releases, and that fixed the previewer problem, but it's a hassle ;f
Additional context
❤

Published the new version to Open VSX

Good day, @prashantvc
Could I bother you to update the Open VSX repository version of the VScode extension to match MS VScode Marketplace ?
Side note: There is a warning about namespaces with a link to the following documentation that might be of interest.
https://github.com/eclipse/openvsx/wiki/Namespace-Access


pick pick #44489 #44894

What problem does this PR solve?
pick #44489 #44894
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[ ] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[ ] No.
[ ] Yes.

Does this need documentation?

[ ] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall


Input validation in separated file

Currently, I use tags for validating inputs with go-playground/validator package, maybe it's cleaner that we move them to validator.go file in each slice.

https://github.com/go-ozzo/ozzo-validation
Input validation in GoLang

u mean use custom validator.go file to replace all the validator package right?
btw, I'm new to this project and really interested to help, do you think this issue is great for me to start contribute?

Maybe we could put it in same command file and replacing validate attribute and validator pkg with ozzo, for example for create_product_command it will be something like this:
type CreateProduct struct {
    ProductID   uuid.UUID
    Name        string
    Description string
    Price       float64
    CreatedAt   time.Time
}

func NewCreateProduct(name string, description string, price float64) (*CreateProduct, error) {
    command := &CreateProduct{
        ProductID:   uuid.NewV4(),
        Name:        name,
        Description: description,
        Price:       price,
        CreatedAt:   time.Now(),
    }
    err := validator.Validate(command)
    if err != nil {
        return nil, err
    }

    return command, nil
}

// using ozzo
func (c *CreateProduct) Validate() error {
    return validation.ValidateStruct(&c,
        validation.Field(&c.ProductID, validation.Required),
        validation.Field(&c.Name, validation.Required, validation.Length(3, 255)),
        ///...
    )
}

If you like, feel free to contribute.

thank you for the tips, let me see what I can do .

you're wellcome :)


Updating Prepare to Dye Plus to 1.3.8

Changelog: ### 1.3.8 (2024-01-14)

@jasperalani is it fully tested?


Unable to double click to edit a memo

Describe the bug
This is achievable in the previous version which is 0.18.1. This change is not mentioned in the release log so I doubt it is a bug rather than intended.
Steps to reproduce

save a memo
double click on it

The version of Memos you're using
0.18.2
Screenshots or additional context
No response

@boojack kindly review this PR related to this issue.

@MehadND why the PR was removed?

@webysther I removed the PR because I either saw somewhere on GitHub or on Discord, where someone or maybe the project owner explained why the double click to edit memo functionality was removed. And at that time it seemed reasonable to remove the PR but if I can get the go from the project owner (@boojack) then I can re-open the PR.

@MehadND Let's do not add double-click to edit, as it's inconsistent with a lot of behaviors and hard to select certain text anymore. We don't want to lose some functionality for convenience.

@MehadND Let's do not add double-click to edit, as it's inconsistent with a lot of behaviors and hard to select certain text anymore. We don't want to lose some functionality for convenience.

Any chance in future add a button direct in header of a note with pen icon to edit?

@MehadND Let's do not add double-click to edit, as it's inconsistent with a lot of behaviors and hard to select certain text anymore. We don't want to lose some functionality for convenience.

Any chance in future add a button direct in header of a note with pen icon to edit?

Well I think if you double click a memo header, it goes to that memo detail page and there you can edit the memo using pen icon. Maybe something you were looking for 🤷🏽♂

I agree with @webysther, a dedicated edit button on the note would be perfect if the double click is no longer an option.

@rb-26 each memo already has its dedicated edit button in the vertical action menu.

@rb-26 each memo already has its dedicated edit button in the vertical action menu.

Yeah, but it is unnessesary clicks when you are editing notes often. Not a huge deal, just a nice to have.


⚠ Bofrost.at has degraded performance

In c0f4293, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1543 ms

Resolved: Bofrost.at performance has improved in 51ca838 after 6 minutes.


Custom Fallback Renditions for Search Results and Details Image

Hi,
Our clients wants to be able to set their own renditions for some of the assets for the Search Results cards and the Image component on the Details page. As such, we created customized InternalRedirectRenditionDispatcherImpl and StaticRenditionDispatcherImpl configs.
We can now upload files named rc-details-thumbnail.png and rc-results-thumbnail.png as new renditions and our configs will pick them up for the Details (Image Component) and Results (Cards) pages respectively.
The only problem is that now assets that they did add these custom renditions to are showing up as broken images. Is there a way to set a fallback on those configs? It just seems like it maps a rendition name to a filename without any options on what to do if that file isn't found...
I know they take regexes so we could create a regex that would satisfy two files, the first of which being the custom one and the second being the standard one, but I'm tentative to trust that the order in which they get resolved in will be consistent. Is that not a realistic concern?
Our current solution is that we have created a custom Processing Profile to create those "custom" renditions automatically as the standard thumbnails that AEM also generates so they always exist and the client can override them as needed. This seems like a risky solution though because if these assets ever get processed again (if someone accidentally triggers a reprocess), it seems as any existing (custom) renditions would get recreated with the default ones.
Any guidance is much appreciated.
Thanks,
Liviu

Following up on this, I have noticed (strangely), that I was indeed able to create a StaticRenditionDispatcherImpl config with an OR in the regex like
"rendition.mappings": [
    "rc-details-thumb=^(rc-details-thumbnail(\\..+)?)|(cq5dam\\.web\\.\\d+\\.\\d+\\..+)$",
]

Where rc-details-thumbnail.png is the rendition that I want it to look for first and the cq5dam one, ie - the part after the OR is what I want it to use as a fallback and it does indeed seem to work. I'm not that well versed in regexes so hopefully this is a consistent and reliable result.
That being said, I also have a InternalRedirectRenditionDispatcherImpl config that is needed to map the AssetRenditionImpl computed property, which is used to find a rendition in places like the card.html results template where

the way the static rendition dispatcher works is

For the asset in question
It gets a list of all the rendition nodes for that asset under .../some-asset.png/jcr:content/renditions
and iterates over each of the children of that renditions node in their natural JCR order (the order youd see in CRXDE)
For each rendition node name, it sees if the regex matches it
It returns the first rendition name that matches the provided regex

So your regex works because your fallback appears to be always checking the rc-... rendition and finding it before continuing on and finding the cq5dam.web... rendition. The order the rendtiions are checked is what its determining which is found first.

I see thanks for that explanation @davidjgonzalez!
What are some better ways to create fallback renditions though, if the one we are specifying isn't found?

@DigiLiviu if its for dispaly on the asset share, you can use the img tags onerror event to set the src to something else if the original SRC doesnt return a 200 .. otherwise there's not a great way to do this :(


DuckDuckGo Images/Videos not blocked

What filter does this issue target?
https://duckduckgo.com/?q=site%3Afacebook.com+facebook&kad=nl_NL&kae=-1&kp=-2&kav=1&kn=1&k1=-1&kaj=m&kam=osm&kay=b&kak=-1&kax=-1&kaq=-1&kap=-1&kao=-1&kau=-1&kg=p
What changes should be made?
Your rule: duckduckgo.com##a.result__a[href*="facebook.com"]:upward(3) doesn't work for images and video's as you can see in the screenshot.
Images from facebook.com are still visible.

Did you write a uBlock filter solving this?
I propose for DuckDuckGo images and videos the following rule:
duckduckgo.com##.tile:has(a[href*="facebook.com"])
Which works, I you can see:

Additional notes
No response

Thanks for reporting this!
Because the copycat presets bring in a lot of urls to match, I am trying to rewrite the filters to be more frugal. I'll try increasing the rules specificity by matching a parent container (to separate text / media results), and use :upward instead of :has (I read it had a better performance profile, but need to find a way to benchmark that). I am deploying the following template, which works for text, image and video results:
  duckduckgo.com##.results a.result__a[href*="{{site}}"]:upward(.result)
  duckduckgo.com##.tile-wrap a[href*="{{site}}"]:upward(.tile)


PayPalPaymentsForm: update and tidy-up

Remove redundent, explicit field definitions; deprecate initial['return_url'], use initia['return'] instead; add new cmd choices.

This looks pretty good to me, just a few flake8 fixes as per the Travis build - https://travis-ci.org/spookylukey/django-paypal/jobs/315857234

Flake8 issues fixed, if you want to have another look.

Thanks so much the for thorough PR!


[Bugfix] add missing database field changetime to ListProduct struct

1. Why is this change necessary?
Because the field is missing in the frontend.
2. What does this change do, exactly?
Add the missing article field changetime to the ListProduct struct and make the field available as $sArticle.update at the frontend.
3. Describe each step to reproduce the issue or behaviour.
.
4. Please link to the relevant issues (if any).
.
5. Which documentation changes (if any) need to be made because of this PR?
.
6. Checklist

[ ] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hey @larsbo,
thanks for your contribution.
Please add an ElasticSearch mapping, see https://github.com/shopware/shopware/blob/5.3/engine/Shopware/Bundle/ESIndexingBundle/Product/ProductMapping.php#L147

ES mapping added :)

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-20735
Please use this issue to track the state of your pull request.

CLA already signed.

Hi @larsbo, your PR was merged in b2ba24ad6873444e2e5f4858094c05a04ea7fa7e. Thank you for your contribution! 💙
Please accept our apology for taking so long. 🙏


Keep failing to import descriptors, not sure why

Hi,
I seem to connect to my full node just fine, get through computing the circulating supply, then when trying to import the descriptors it fails. Here's a small snippet of the terminal:
[GIN] 2022/07/24 - 19:16:41 | 200 |      30.056µs |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:41]  INFO worker: #RunTheNumbers successful                     actualSupply=19102167.1778084 BTC expectedSupply=19102375 BTC height=746380
[2022/07/24 - 19:16:42] ERROR Failed to import descriptor                   descriptor=<descriptor for my accounts>

*** more failed attempts at different descriptors, each tried twice ***

[2022/07/24 - 19:16:42] ERROR worker: Failed while importing descriptors            error=importmulti RPC failed
[2022/07/24 - 19:16:42]  INFO Shutdown server: in progress
[2022/07/24 - 19:16:44] ERROR Failed to query status                        err=bitcoind unreachable: -18: Requested wallet does not exist or is not loaded
[GIN] 2022/07/24 - 19:16:44 | 200 |  1.928707502s |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:44] ERROR Failed to query status                        err=bitcoind unreachable: -18: Requested wallet does not exist or is not loaded
[GIN] 2022/07/24 - 19:16:44 | 200 |  1.362037923s |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:45] ERROR Failed to query status                        err=bitcoind unreachable: -18: Requested wallet does not exist or is not loaded
[GIN] 2022/07/24 - 19:16:45 | 200 |  1.312411602s |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:46] ERROR Failed to query status                        err=bitcoind unreachable: -18: Requested wallet does not exist or is not loaded
[GIN] 2022/07/24 - 19:16:46 | 200 |  772.110664ms |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:46]  INFO Unloaded wallet successfully                  wallet=satstack
[2022/07/24 - 19:16:46] ERROR worker: Failed to query wallet state                  error=the client has been shutdown
[2022/07/24 - 19:16:46]  INFO worker: Shutdown worker: done
[2022/07/24 - 19:16:46] ERROR Failed to query status                        err=bitcoind unreachable: -18: Requested wallet does not exist or is not loaded
[GIN] 2022/07/24 - 19:16:46 | 200 |  469.477654ms |       <IP_ADDRESS> | GET      "/blockchain/v3/explorer/status"
[2022/07/24 - 19:16:47]  INFO Shutdown server: done

[Process completed]

I've tried this about 3 times now it's repeated the above. Not sure why it's happening. It also seems to be in debug mode when ran? This differs from a screenshot I saw of the program running in a terminal. I'm running this in MacOS, and followed the instructions from Ledger Live.
PS It would be helpful to bypass computing the circulating supply just so I could try importing the accounts, seen from an issue someone raised earlier   #41 :)
Let me know if you need any more info or have any questions!
Oh, just trying to import 2 native segwit accounts.
Appreciate it!
-Michael

hi @MikePython96 any error in your bitcoind logs?

hi @MikePython96 any error in your bitcoind logs?

Okay so I guess bitcoin daemon isn't installed with Bitcoin Core, so I installed that lol.
But how would I see bitcoind with lss.exe running? Whenever I run bitcoind, lss shuts down with a 401 error saying it can't reach bitcoind.

hi @MikePython96 any error in your bitcoind logs?

Okay so I guess bitcoin daemon isn't installed with Bitcoin Core, so I installed that lol. But how would I see bitcoind with lss.exe running? Whenever I run bitcoind, lss shuts down with a 401 error saying it can't reach bitcoind.

It should be...

hi @MikePython96 any error in your bitcoind logs?

Okay so I guess bitcoin daemon isn't installed with Bitcoin Core, so I installed that lol. But how would I see bitcoind with lss.exe running? Whenever I run bitcoind, lss shuts down with a 401 error saying it can't reach bitcoind.

It should be...
You just need to find the logs

I found the log file for both the wallet and bitcoind, but the wallet was blank and bitcoind didn't show any errors.
However, I did notice a transaction that popped up that I just made, so it made me think it actually did load my Ledger wallet... so I rescanned the blockchain and all my transactions are now loaded.
Going back to lss.exe, the same errors pop up, but I noticed it's only hardened accounts and the purpose in the derivation path is 86', not 89'.
I do only have 2 accounts on my ledger wallet, and I said to import up to 4... could this be why the errors are popping up? I wouldn't think this would be causing errors.
But anyway, although I haven't figured this out, my problem seems to have gone away/I just needed to rescan in order to import my two bitcoin accounts. Wish I could figure out what exactly is going on but I need to move on with other things 😩
Appreciate the help!!
-Michael

Glad your problem is gone :) feel free to close the issue.
No problem!


MudTab active area becomes top half of tab with MudGrid/MudItem content

Things to check

[X] I have searched the existing issues for this bug
[X] To rule out a caching problem I made sure the bug also happens in an incognito tab

Bug type
Component
Component name
MudTabs
What happened?
When you add a MudGrid and MudItem as contents of a MudTab panel, suddenly only the top half of the tab becomes becomes active (is highlighted when you hover). This makes it harder for users to select the tab, and is easily replicated with this snippet. The problem occurs in both v7 and v8 preview 6.

Expected behavior
When you hover over any area in the tab, it highlights and becomes selectable.
Reproduction link
https://try.mudblazor.com/snippet/GacoPclspUBCOzKd
Reproduction steps

Run the above snippet
Hover with mouse over either tab. It will only become active when you hover over the top half.

Relevant log output
No response
Version (bug)
7.x and 8.0 preview 6
Version (working)
No response
What browsers are you seeing the problem on?
Edge
On which operating systems are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Can reproduce with your empty grids but not with actual data https://try.mudblazor.com/snippet/QkwIlmFMgOYqSHde

Works correctly if your Grid has content https://try.mudblazor.com/snippet/wOQovQFCgYMNZleC

@Anu6is That's not exactly true. It works correctly because you added "pa-6" to your panel class. If you take the "pa-6" out of your sample, then only the top half of the tab is active as with my example. I should have pointed that out initially--there IS a way to make the entire tab active, but ONLY if you move the MudGrid content away from it with a pa-6 or pt-6. Using pa-3 makes it active about a quarter way up the tab, and pa-0 makes only the top half active.

Duplicate of #9477. See https://github.com/MudBlazor/MudBlazor/issues/9477#issuecomment-2246550456 for workaround.


Fix cache problem when going from Python 2 to 3

See https://github.com/nvbn/thefuck/issues/404#issuecomment-156816429.

Thanks!


Client provisioning updates

Makes a few changes to the client provisioning playbook:

removes the grpc dependency, since we don't need to install from source anymore (yay!)
adds a rocksdb playbook to install rocks into /usr

i'd prefer /usr/local but I had trouble getting the pyrocksdb package to find it when installed there

makes transactor-client-provisioning.yml depend on rocks playbook, install pyrocksdb to system pip
adds ipfs role from ansible galaxy to transactor-client-provisioning.yml

The last one means that you'll need to run ansible-galaxy install insanity54.ipfs on your local machine after this is merged.
Also, one thing to note is that the ipfs daemon runs as the ipfs user (created by the ansible role), so if you shell in and try to run e.g. ipfs cat some-hash it will complain at you.  Everything seems to work via the HTTP api though, which is what we're using in the client.  You can su to the ipfs user if you need to run ipfs commands from the cli.
@vyzo want to try this out tomorrow and pull it in?

Also, vyzo, if you find that ipfs is slowing down ingestion or eating up too much cpu tomorrow, you can disable it during ingestion with the --disable-ipfs cli flag.  It was switched to "on by default" in https://github.com/mediachain/mediachain-client/pull/65

Uhm /usr installation is rather rude -- can we fix that one please?

ugh, there's a problem with the ipfs role; apparently since ansible 2.1 it's much harder to become an unprivileged user, so it fails when running ipfs init as the ipfs user.  I thought I had fixed that yesterday, but apparently not.

so the ipfs thing is an issue with ipfs, not ansible: https://github.com/ipfs/go-ipfs/issues/2748
ipfs init fails if stdin is not a tty 😞  it's fixed in master but not released yet.
I think I'll just remove the ipfs role for now, and we can revisit later.

 Can you try script --return -c "ipfs init" /dev/null?

nice, that seems to work 👍  I'll push an update in a sec.


HTML syntax on prettier whitespace sensitivty

The html below would cause syntax for the

The screen capture:

The syntax would be correct if the closing  has no whitespace

This kind of formatting of not losing whitespace is now a default in prettier ( https://prettier.io/blog/2018/11/07/1.15.0.html#whitespace-sensitive-formatting ).
In Vue, the official cli program would generate component where the

The screen capture:

The syntax would be correct if the closing  has no whitespace

This kind of formatting of not losing whitespace is now a default in prettier ( https://prettier.io/blog/2018/11/07/1.15.0.html#whitespace-sensitive-formatting ).

The example has a line break, but the same happens when the closing tag
is , while it looks OK with .
Jorge, can you fix this?
--
Contrary to popular belief, Unix is user friendly.
It just happens to be selective about who it makes friends with.
-- Dave Parnas
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

This issue applies to every region inside if !exists("html_no_rendering"). In this case, on the line 194. When you add a space before the <, me=e-4 is wrong, and it would need to be me=e-5 for it to work. I don't know if it's possible to do something like me=e-[end-match-length], so that there are no highlighting problems, I didn't find such an option in the documentation. Also I didn't code that part and I don't know if there is a better way to highlight.

Jorge, can you fix this?

As you've seen, I tried without sucess. I won't work more on this because I lack the time, but I hope that this comment will help and I'm happy to review any patch which aims to solve this issue.

I took the line 194, where the htmlLink region is defined:
syn region htmlLink start="\_[^>]*\<href\>" end=""me=e-4 contains=@Spell,htmlTag,htmlEndTag,htmlSpecialChar,htmlPreProc,htmlComment,htmlLeadingSpace,javaScript,@htmlPreproc

And changed end=""me=e-4 to end=""me=s-1. After this the OP's html code highlighting worked great.
As my knowledge of html is very poor, I'm not able to do proper testing. So I cannot tell if this change breaks anything. What do you think, @jorgesumle or anybody? Might this be the way to go?
Tom

What do you think, @jorgesumle or anybody? Might this be the way to go?

Sorry, I didn't read the documentation carefully enough. Sure, it does exactly what we want and it's more flexible than the previous code where the tag length was needed to proper highlight and just one space could mess up everything. Thanks a lot for the fix @tom-m!
Could you please send a pull request to the repository (or email me using git request-pull or a patch, I don't really mind)? When it's ready, I'll send the patch to Bram so that we can solve this issue.

After more than a week, I still have no answer from you, @tom-m. If you don't answer in two days, I'll assume you're not working on the patch and I'll work on it to fix this issue.

@jorgesumle
Sorry for answering late, Jorge. I was not watching this thread too closely. Anyway, as I stated before, I'm no HTML guru. I cannot tell for sure where does HTML allow whitespace. So I didn't feel to be in a position to propose changes to the syntax file myself. Thank you for coming up with proper fix for this issue.
Tom

Don't worry, @tom-m, thanks for your help!

I cannot tell for sure where does HTML allow whitespace.

According to the W3C HTML Recommendation: "After the [end] tag name, there may be one or more space characters".

closed by #4219


Improve documentation of service bus ServiceBusReceiveMode and ServiceBusMessageState

Since there classes are subclasses of str and int respectively the docs include all the str and int methods, that don't make much sense in this context.
Webpage:

https://azuresdkdocs.blob.core.windows.net/$web/python/azure-servicebus/latest/azure.servicebus.html#azure.servicebus.ServiceBusMessageState
https://azuresdkdocs.blob.core.windows.net/$web/python/azure-servicebus/latest/azure.servicebus.html#azure.servicebus.ServiceBusReceiveMode

edit: it applies also to ServiceBusSubQueue https://azuresdkdocs.blob.core.windows.net/$web/python/azure-servicebus/latest/azure.servicebus.html#azure.servicebus.ServiceBusSubQueue and many names in the https://azuresdkdocs.blob.core.windows.net/$web/python/azure-servicebus/latest/azure.servicebus.management.html# module

Thank you for the feedback @kashifkhan . We will investigate and get back to you asap.

Hi @CaselIT we have a pr out to fix this issue, thanks for bringing this to our attention.

thanks!


Doesn't run on OS X10.11.1 Beta (15B30a)

This version doesn't open on this version OS X.

Thank you for reporting this. At which point the program failed? Do you see any error message?
Thank you for the additional information.

Unfortunatelly, it doesn't show any error message. It crash after few seconds. I can say that RC1 runs great on this OS X build.

can't find 15B30a. Is it sha? I tried the updated version  8b4a1d on Mac it start fine.

@FigaCZ Can you please clarify what version of OpenRefine you are attempting to run and which OS X 10.11.1 Beta you are running (there have been four so far).
My first inclination, if OpenRefine runs on released versions of OS X, is to report this as an OS X bug to Apple.


/Users/ice/.oh-my-zsh/custom/plugins/zsh-autosuggestions/zsh-autosuggestions.plugin.zsh:source:1: no such file or directory: /Users/ice/.oh-my-zsh/custom/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh^M

/Users/ice/.oh-my-zsh/custom/plugins/zsh-autosuggestions/zsh-autosuggestions.plugin.zsh:source:1: no such file or directory: /Users/ice/.oh-my-zsh/custom/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh^M

macOS: 10.13.6
zsh --verison
zsh 5.8 (x86_64-apple-darwin17.7.0)

I installed it
git clone git://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions

I have the same problem.
WSL 2.0
lsb_release -a                                                                                No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

zsh version
zsh 5.4.2 (x86_64-ubuntu-linux-gnu)

I had this same issue in WSL2, it was due to CLRF line endings on the files. Once I switched them to LF it worked fine.

I had this same issue in WSL2, it was due to CLRF line endings on the files. Once I switched them to LF it worked fine.


Not working in IPv6 networks

Hi,
App become irresponsible and then crashes when I try to use scanner in IPv6 network
self.device.ipAddress

looks like 169.254.xxx.xxx
[LANProperties getAllHostsForIP:self.device.ipAddress andSubnet:self.device.subnetMask];

return an array of 65534 IPs
from  <IP_ADDRESS> to <IP_ADDRESS>
It will be great if you can fix this =)
If you don't know how to test in ipv6 network look here
Regards

Also, maybe this will help
#include <arpa/inet.h>
#include <resolv.h>

res_state res = malloc(sizeof(struct __res_state));

int result = res_ninit(res);

if(result == 0){
    // pass

    int nscount = 0;
    union res_sockaddr_union u[MAXNS];
    nscount = res_getservers(res, u, MAXNS);
    NSLog(@"nscount = %d", nscount);
    int i;
    for(i = 0; i < nscount; i++){
        if(u[i].sin.sin_family == PF_INET){
            NSLog(@"IPv4");
            char straddr[INET_ADDRSTRLEN];
            inet_ntop(PF_INET, &u[i].sin.sin_addr, straddr, sizeof(straddr));
            NSString *s = [NSString stringWithCString:straddr encoding:NSASCIIStringEncoding];
            NSLog(@"%@", s);
        }
        if(u[i].sin6.sin6_family == PF_INET6){
            NSLog(@"IPv6");
            //struct sockaddr_in6 *addr6 = (struct sockaddr_in6 *)&u[i].sin6;
            char straddr[INET6_ADDRSTRLEN];
            inet_ntop(PF_INET6, &u[i].sin6.sin6_addr, straddr, sizeof(straddr));
            NSString *s = [NSString stringWithCString:straddr encoding:NSASCIIStringEncoding];
            NSLog(@"%@", s);
        }
    }

}else{
    // fail
}

https://hirooka.pro/?p=6169

Hi,
MMLanScan doesn't support IPV6. It's entirely different concept and that's why you get an IP 169.* It cant get it's subnet and its assumes that is /1, that's why is crashing. I should add a case to not scan IPv6 networks. If you implement it feel free to send me a pull request


Add Shapeshift.io support to wallet

Hooray, https://shapeshift.io/#/coins now supports Siacoin!
With a proper plugin/enhancement, users will be able to buy storage space directly with Bitcoin! You could even let people withdraw what they make from hosting directly into BTC.

This should be easy, just add a new tab with an iframe. In the iframe you load this page:
https://shapeshift.io/shifty.html?destination=[Siacoin wallet address]&output=SC
And you fill the destination field with a generated address from the siad API

We probably will not create this plugin ourselves (to avoid favoritism), but it sounds like a great opportunity for a community member to write a plugin.

Closing this issue since we're not planning on adding shapeshift support directly to the wallet (as @lukechampine mentioned- favoritism).

I'll try my hand at a plugin this evening. The plugin tutorial is really good, I hadn't even looked at it before.

It's worth noting that the plugin tutorial is outdated, the IPC stuff has been replaced with SiaAPI and notifications have been removed. Plugins are still just webpages. You can read about SiaAPI here: https://github.com/NebulousLabs/Sia-UI/blob/master/doc/spec/PluginAPI.md.


Proposal: Mitre Att&ck Checker library for Falco Rules

Motivation
The rules must be both syntactically and grammatically correct and should evaluate to true during successful end-to-end tests. Furthermore, it needs to accurately detect the intended cyber threats, specifically the Tactics, Techniques, and Procedures (TTPs), against the Mitre ATT&CK framework. Both the community and the Falco experts will benefit from the falco mitre checker module to audit the default rules or custom rules against the STIX2 data from Mitre CTI.
Feature
Develop a library to check the compliance of the Falco rules against the Mitre ATT&CK Framework. This library will provide to Falco experts and Falco users a way to check default and custom rules for Mitre ATT&CK extra tags.
The library will use STIX from the OASIS standards. Structured Threat Information Expression (STIX™) is a language and serialization format used to exchange cyber threat intelligence (CTI) :

STIX CTI documentation

Leveraging STIX, the library will fetch the ATT&CK® STIX Data from MITRE ATT&CK repositories using the python-stix2 library implemented by OASIS:

ATT&CK STIX Data repository
Python STIX2 repository

The choice of a library is motivated by the packaging of a python code to integrate it into wider Falco implementations. More precisely, the library can be used :

by the rules_overview_generator.py script
Falco users and experts to check their falco rules files
Other falco components that need to check the validity of rules files

Design Choice
To benefit from python-stix2, the library will be developed in Python 3.11 according to the last stable version released on this date.
The library should take as inputs one or more Falco rules files and run a validity checker implementation on each file.
The library could be named mitre_checker and could be located in the build directory :
.
├── build
    ├── checker
    ├── mitre_checker
    └── registry

The implementation consists in :

Fetching the enterprise-matrix STIX2 data from the Mitre CTI repository
load the data in memory
load the falco rules in memory
look for the falco rules that contains extra tags that concerns mitre information. These tags should contain mitre_<mitre-phase-name> (backward compatible) and/or <technique id> (backward compatible).
verify the validity of the relation between the technique and the mitre phase if a <technique id> is detected in the extra tags.

Expected Output
The library should provide an in-memory report, in form of a model, which gather information about errors in the falco rules files that concern Mitre ATT&CK extra tags.
The report can be dumped to be stored on disk (optional).
Packaging
The library can be packaged in a wheel file in the first place. In this way, it could be pushed in public pypi repositories. Otherwise, a developper can easily install it in any python environment or build it again from scratch.
Extra packaging like a binary file or a container can be considered for further integrations.
Alternatives
Mitre ATT&CK framework is growing as a 'de facto' standard for TTPs knowledge and studies. I do not know another framework to consider it as an alternative.
Mitre CTI sharing chose STIX2 standard to maintain its Mitre ATT&CK data. A known alternative is OpenCTI but it has to be considered as a set of tools based on STIX2 rather than an alternative to STIX.
Additional context
This initiative was discussed from the issue #84, in wip: #76 and during discussions on slack.

Thanks a bunch @IceManGreen!
Initial feedback:
Python is a good and appropriate choice here for several reasons: (1) It is well-suited for small-scale Data Science projects like this one due to its strong libraries, (2) Python facilitates broad adoption and reusability among many adopters, making it easier for others to use, and (3) since it is not crucial for deployment purposes, there is no need for critical optimizations.
STIX2 data from Mitre CTI seems suitable from my perspective. Thanks for outlining alternatives considered.
Great implementation details! The high-level outline for the new Python module looks great, and we can provide further guidance during the PR review process. As suggested, please refactor and integrate the existing rules overview Python script into the new module. I can assist with the refactoring process since we are currently working on the new rules maturity and adoption framework
re packaging: Let's consider adopting an approach similar to https://github.com/falcosecurity/kernel-crawler to ensure consistency within The Falco Project. Let me think more about the best directory outline, as this repository serves multiple purposes. However, I don't believe we should create yet another repository.
re goals

verify the validity of the relation between the technique and the mitre phase if a  is detected in the extra tags.

Can we outline stronger benefits?
For example, since the Mitre tags are manually added by experts, errors can occur. This module serves as a cross-check to ensure that we have tagged a rule with the best available Mitre Attack TTPs. Furthermore, it provides additional educational material for adopters.

@falcosecurity/rules-maintainers kindly asking for additional feedback. Are we moving forward with the proposed plan?

Hello @incertum !

Can we outline stronger benefits?
For example, since the Mitre tags are manually added by experts, errors can occur. This module serves as a cross-check to ensure that we have tagged a rule with the best available Mitre Attack TTPs. Furthermore, it provides additional educational material for adopters.

This a very interesting and complex question.
@hkonduri and I talked about it on Slack and we think, like you, that the discussion should go further between all of us. Like he suggested on Slack we could, for example, base some rules conditions on Mitre ATT&CK datasources.
But it raises other questions like :

Are we able to bind syscalls with Mitre ATT&CK datasources using Falco ? For exemple, for the datasource DS0014 Pod creation, are we able to link the proper syscalls to a pod creation ? If yes, how to cross-check the syscalls to the datasource too ?
For the following Mitre ATT&CK datasources defined for T1610 Deploy Container:

"x_mitre_data_sources": [
        "Pod: Pod Modification",
        "Container: Container Creation",
        "Pod: Pod Creation",
        "Application Log: Application Log Content",
        "Container: Container Start"
      ]

are we sure that if we detect all the syscalls for all these datasources, we can consider that it is surely a event for T1610 ? How can we cross-check this ?

If we consider that all the syscalls above are featuring a pod creation, should we modify the Falco rules conditions (or maybe macros ?) to also include the detection of the datasources to add the proper Mitre ATT&CK extra tags ?

What is your opinion about it ?
If anyone has hints or suggestions, please comment !

@IceManGreen 🤯 these are all excellent questions and great thinking btw!
I hope we get to a solution that would help us more automagically craft rules and link them to Mitre. As a first step, perhaps we can keep the immediate benefits more scoped?
Here are some additional thoughts:

Initially we should continue tagging the rules manually to the best of our ability
The checker you are proposing helps avoid some human mistakes
In addition, I see tremendous educational value in such a tool as we could enhance the rules overview document with additional links and resource that may inspire adopters and subsequently we can improve the robustness of rules and ensure we have the right rules to detect xyz.

@falcosecurity/rules-maintainers kind bump to give everyone the opportunity to share their feedback.
@IceManGreen let's give it another week to allow for feedback, after that I would make the judgement call to get started if no one objects. I am fully supportive of this proposal and will help with the review process the best I can.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle stale
/remove-lifecycle rotten
We still want this. I apologize this is taking so long. Since we still have a few ongoing license discussions with the CNCF, we just want to be prudent before proceeding. That being said,  I am still fully supportive of this proposal.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

Related to #181 (merged).
The next step should involve the integration of the former implementation in a CI/CD job in order to validate (at least) the stable rules. Maybe even the incubating and sandbox ones ?
This proposal can be closed, or maybe kept alive to discuss about the CI/CD job ?

We can close this and maybe open a new one for the CI integration.
FYI I am back to having more available and we can start working on it if you want. I'll also check how else we can leverage the new module. Thank you!

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

@leogr: Closing this issue.

In response to this:

We can close this and maybe open a new one for the CI integration.

:+1:
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.


Agent output streams

I just saw that AutoGen is planning to add an output streams feature beyond console output.
https://github.com/microsoft/autogen/pull/1290#discussion_r1460794605
This would be great if crewAI would provide a similar feature.

Interesting, curious to hear more about the use cases so we can get something that is great for those, anything comes to mind?

One use case would be to stream, agent result to a web frontend.
Another could be to have really distributed agents working together.

I agree, i'd like to send answers via WebSockets to a web client.

I also want to be able to stream live output to a frontend.

Currently I found a workaround to capture terminal output while crew.kickoff() is working with subprocess and to stream the stdout. It’s ugly and gets unsupported characters, but at least can stream.

Currently I found a workaround to capture terminal output while crew.kickoff() is working with subprocess and to stream the stdout. It’s ugly and gets unsupported characters, but at least can stream.

please, share the code
i'm currently doing it like this and it sucks
from io import StringIO  # Python 3
import sys

# Create the in-memory "file"
temp_out = StringIO()

# Replace default stdout (terminal) with our stream
sys.stdout = temp_out

print("This is going in to the memory stream")


Allow unabridged API to exclude different kinds.

Fixes #64.

Default to trimming dir and file.
Utilize :orphan: page attribute (to be continued?).

@Coderik @TheMaverickProgrammer it ended up being possible to ignore warnings from sphinx about documents not included in any toctree (:orphan: page attribute).  This is Option 1 from here since we can get 0 warnings https://github.com/svenevs/exhale/issues/64#issuecomment-494544925

So basically

By default, dir and file will now be excluded from the full api listing.
You can optionally choose more kinds to exclude if you desire.

Note that this does not enable removing a single element from there.  That will be possible in a (somewhat distant) future.  Custom writers will happen at some point, but not until a few other bigger problems can be fixed.
Would you be willing to test this and make sure it does what you want it to?  You can install it (ideally in a virtual environment ;)) with pip install git+https://github.com/svenevs/exhale.git@fix/unabridged-bury-files-directories

This is awesome

Tried adding your suggested changes and cant get it to build on RTD.
Link
Raw text
I added git+https://github.com/svenevs/exhale.git@fix/unabridged-bury-files-directories to my requirements.txt file.
I reverted back to the last version.

breathe 4.13.0.post0 has requirement Sphinx>=2.0, but you'll have sphinx 1.8.5 which is incompatible.

I need to update the quickstart / will be adding this in release notes, sorry this wasn't very clear.  Since you are using python3.7 you should require sphinx>=2.0.  In your docs/requirements.txt:
+ sphinx>=2.0
+ breathe>=4.13.0.post0
- breathe

Sphinx 2 drops python 2 support, breathe 4.13.0.post0 drops Sphinx 1.x support.
So for projects that have a specific need to remain on py2, they must pin sphinx<=1.8.5 and breathe<=4.12.0.
Thank you for testing, that is really helpful!

I'm not getting those sphinx warning anymore but it still fails silently at the same step: link.

I see you forked the project... It's pretty big. The branch I'm using on Read The Docs is #windows-linux-merge.
The code on this branch doesn't run correctly so dont expect to play it. It will compile and it is documented.
If you do want to play the game, stick to #master branch as it is stable.

https://readthedocs.org/api/v2/build/9133256.txt
It seems to be working now. I do see some comments in the conf.py file I don't remember writing. Did you update the file? :D

I do see some comments in the conf.py file I don't remember writing

Haha yeah I added some notes in there because I will probably break the hacks that PR added in v1.0, depending on how the writer API comes in.  Meaning you'll trade the hacks in favor of using the writer API.
This branch just got deleted (so you'll need to update your requirements since it's installing a non-existent branch), but this is released on PyPI right now in v0.2.3 :)  AKA require exhale>=0.2.3.
Sorry for delay on packaging this, thanks again for helping test it!


Remove code coverage and add failed tests logging

Description
AB#4232 Make some enhancement for the Test Stability pipeline

Remove the code-coverage reporting, will save ~ 5 mins per iteration, and we have 100 iterations
Publish tinylicious logs (identifiable for the specific iteration), will be easier to troubleshoot when the tinylicious tests fail

Successful testing has been conducted on the test branch, confirming that the implementation is functioning correctly.

Previously, I had considered including a task to publish Tinylicious logs as the same as build-client package pipeline. However, I discovered that each Tinylicious log consumes 74 MB of space, and uploading 100 logs would be too resource-intensive. Additionally, I explored the possibility of only uploading Tinylicious logs if any related tests failed, but I found this to be difficult to implement given the current YAML file design (I do not suppose we have any feasible command to detect if one specific test fails in a past task). Instead, I decided to log error messages to the console if any tests failed.
@alexvy86 Do you have any suggestions or advice on this matter? For instance, do you think it would be feasible to compress 100 Tinylicious logs using zip?

Compressing all the logs seems like a good idea but might still generate too big an artifact. I gzip-ed a 74Mb tinylicious log and it became a 9Mb file; 900Mb worth of tinylicious logs still feels like a lot.
But I think we could upload the tests conditioned on test failures with a bit of extra logic:

Add the name property (docs) to the step that runs the tests. Probably something like ${{ replace(taskTestStep, ':', '_' ) }} (colons are not allowed, only [-_A-Za-z0-9]).
For the step that renames the tinylicious log file, add dependsOn: ${{ replace(taskTestStep, ':', '_' ) }} and condition:  and(failed(), contains(taskTestStep, 'tinylicious')). I think that combination would only make it run if the step it depends on (the one that runs the tests) fails (so only when there were issues running the tests) and only in the case of tinylicious tests.
For the step that uploads the renamed file, add dependsOn: <name-of-the-step-from-the-previous-bullet>, so it only runs if that previous step succeeded.

Also, it'd be good to see a test run where the tinylicious tests fail, to confirm that the step that logs the failed tests is working correctly. I see the test run where you added that logic passed all tests.

Compressing all the logs seems like a good idea but might still generate too big an artifact. I gzip-ed a 74Mb tinylicious log and it became a 9Mb file; 900Mb worth of tinylicious logs still feels like a lot.
But I think we could upload the tests conditioned on test failures with a bit of extra logic:

Add the name property (docs) to the step that runs the tests. Probably something like ${{ replace(taskTestStep, ':', '_' ) }} (colons are not allowed, only [-_A-Za-z0-9]).
For the step that renames the tinylicious log file, add dependsOn: ${{ replace(taskTestStep, ':', '_' ) }} (i.e. the same expression that went in the name in the previous bullet point) and condition:  and(failed(), contains(taskTestStep, 'tinylicious')). I think that combination would only make it run if the step it depends on (the one that runs the tests) fails (so only when there were issues running the tests) and only in the case of tinylicious tests.
For the step that uploads the renamed file, add dependsOn: <name-of-the-step-from-the-previous-bullet>, so it only runs if that previous step succeeded.

Also, it'd be good to see a test run where the tinylicious tests fail, to confirm that the step that logs the failed tests is working correctly. I see the test run where you added that logic passed all tests.

I have made an update to the yaml file that now allows the tests to be uploaded based on test failures. To demonstrate this functionality, I ran a test on my personal test branch where I intentionally modified an e2e test: https://dev.azure.com/fluidframework/internal/_build/results?buildId=160740&view=results. As a result, five different artifacts were successfully uploaded. This serves as a clear example of how the update effectively handles test failures and triggers the appropriate artifact uploads.


HWY-181: Simplify State endorsement fields.

Also check on pre-validation whether endorsements are empty, so we don't create empty entries in those maps.
https://casperlabs.atlassian.net/browse/HWY-181

bors merge

bors r-

bors merge


fix the CI status badge

The link was pointing to "?branch=master" while it needed to point to "?branch=main". (I'd taken this link from the Pebble README.md which does have a master branch).

This change is

Did you forget to "approve"?


Utilize GitHub API to perform commits

The gato attack features currently push changes using git commands. Gato will not work against organizations that require SSH certificate authentication for clones and push operations. It is possible to use the GitHub API to create a new branch, commit to that branch, and then delete it. This will bypass the SSH authentication requirement entirely.

Currently tracked in update/api_only_wf_attack. Confirmed working but not thoroughly tested.

Feature is in the development branch for enumeration and workflow attack. Fork PR attack still uses git commands.

After looking at real-world use cases, it does not make sense to use the API to perform the force-push and close technique. It is a lot of extra work for little since
the log footprint is the same.
Any complex attacks abusing a fork PR are best conducted manually, and this feature is mainly intended to support pentesters wanting to demonstrate impact quickly.


[RFE] Default to use kube-ca for authorized endpoint when FQDN is specified without a cacert

What kind of request is this (question/bug/enhancement/feature request):
enhancement
Steps to reproduce (least amount of steps as possible):

Setup Layer4 Load Balancer for downstream cluster
Specify domain name of load balancer in sans  https://rancher.com/docs/rke/latest/en/config-options/authentication/
Specify FDQN in authorized endpoint
Save
Download the kubeconfig file and run kubectl --kubeconfig /custom/path/kube.config --context <CLUSTER_NAME>-fqdn get pods

Result:
kubectl failed due to no cacert in authorized endpoint
Other details that may be helpful:
It is useful when setting up the downstream cluster with automation tools
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): Rancher v2.5.3
Installation option (single install/HA): HA

Please check for duplicate with https://github.com/rancher/rancher/issues/26986