kao.add_tag(f"CrowdStrike:adversary:branch: {actor_branch}") AttributeError: 'NoneType' object has no attribute 'add_tag'

python3 misp_import.py -f -nb -d --fullmonty
[2023-03-31 19:23:50,002] INFO     misp_tools    MISP Import for CrowdStrike Threat Intelligence v0.6.8
[2023-03-31 19:23:50,002] INFO     config  CHECK CONFIG
[2023-03-31 19:23:50,003] DEBUG    config  client_id                                   value redacted, check config file
[2023-03-31 19:23:50,003] DEBUG    config  client_secret                               value redacted, check config file
[2023-03-31 19:23:50,003] DEBUG    config  crowdstrike_url                             US1
[2023-03-31 19:23:50,003] DEBUG    config  api_request_max                             5000
[2023-03-31 19:23:50,003] DEBUG    config  api_enable_ssl                              True
[2023-03-31 19:23:50,003] DEBUG    config  reports_timestamp_filename                  lastReportsUpdate.dat
[2023-03-31 19:23:50,003] DEBUG    config  indicators_timestamp_filename               lastIndicatorsUpdate.dat
[2023-03-31 19:23:50,003] DEBUG    config  actors_timestamp_filename                   lastActorsUpdate.dat
[2023-03-31 19:23:50,003] DEBUG    config  init_reports_days_before                    365
[2023-03-31 19:23:50,003] DEBUG    config  init_indicators_minutes_before              20220
[2023-03-31 19:23:50,003] DEBUG    config  init_actors_days_before                     730
[2023-03-31 19:23:50,003] DEBUG    config  reports_unique_tag                          CrowdStrike: REPORT
[2023-03-31 19:23:50,004] DEBUG    config  indicators_unique_tag                       CrowdStrike: INDICATOR
[2023-03-31 19:23:50,004] DEBUG    config  actors_unique_tag                           CrowdStrike: ADVERSARY
[2023-03-31 19:23:50,004] DEBUG    config  reports_tags                                att:source="Crowdstrike.Report"
[2023-03-31 19:23:50,004] DEBUG    config  indicators_tags                             att:source="Crowdstrike.Indicators"
[2023-03-31 19:23:50,004] DEBUG    config  actors_tags                                 att:source="Crowdstrike.Actors"
[2023-03-31 19:23:50,004] DEBUG    config  unknown_mapping                             CrowdStrike:indicator:galaxy: UNATTRIBUTED
[2023-03-31 19:23:50,004] DEBUG    config  unattributed_title                          Unattributed indicators:
[2023-03-31 19:23:50,004] DEBUG    config  indicator_type_title                        Indicator Type:
[2023-03-31 19:23:50,004] DEBUG    config  malware_family_title                        Malware Family:
[2023-03-31 19:23:50,004] DEBUG    config  log_duplicates_as_sightings                 True
[2023-03-31 19:23:50,004] DEBUG    config  misp_url                                    https://3samisp
[2023-03-31 19:23:50,004] DEBUG    config  misp_auth_key                               value redacted, check config file
[2023-03-31 19:23:50,004] DEBUG    config  crowdstrike_org_uuid                        ca4f4b5d-db04-4a5e-a6de-e60636dc01be
[2023-03-31 19:23:50,004] DEBUG    config  misp_enable_ssl                             False
[2023-03-31 19:23:50,004] WARNING  config  misp_enable_ssl                             SSL is disabled for MISP API requests
[2023-03-31 19:23:50,004] DEBUG    config  ind_attribute_batch_size                    2500
[2023-03-31 19:23:50,004] DEBUG    config  event_save_memory_refresh_interval          180
[2023-03-31 19:23:50,004] DEBUG    config  max_threads                                 16
[2023-03-31 19:23:50,004] DEBUG    config  miss_track_file                             no_galaxy_mapping.log
[2023-03-31 19:23:50,004] DEBUG    config  galaxies_map_file                           galaxy.ini
[2023-03-31 19:23:50,004] DEBUG    config  tag_unknown_galaxy_maps                     True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_kill-chain                        True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_information-security-data-source  True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_type                              True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_iep                               False
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_iep2                              True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_iep2_version                      False
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_tlp                               True
[2023-03-31 19:23:50,004] DEBUG    config  taxonomic_workflow                          True
[2023-03-31 19:23:50,892] INFO     config  No configuration errors found (1 warning)
[2023-03-31 19:23:50,892] INFO     config
[2023-03-31 19:23:50,892] INFO     config  ____ _  _ ____ ____ _  _ ____    ___  ____ ____ ____ ____ ___
[2023-03-31 19:23:50,892] INFO     config  |    |__| |___ |    |_/  [__     |__] |__| [__  [__  |___ |  \
[2023-03-31 19:23:50,892] INFO     config  |___ |  | |___ |___ | \_ ___]    |    |  | ___] ___] |___ |__/
[2023-03-31 19:23:50,892] INFO     config
[2023-03-31 19:23:52,593] INFO     processor/main       BEGIN ADVERSARIES IMPORT
[2023-03-31 19:23:52,593] INFO     processor/main       Start importing CrowdStrike Adversaries as events into MISP (past 730 days).
[2023-03-31 19:23:52,593] INFO     processor/main       Retrieving all adversaries.
[2023-03-31 19:23:53,656] INFO     processor/main       Got 54 adversaries from the Crowdstrike Intel API.
[2023-03-31 19:23:55,700] WARNING  processor/thread_0   Adversary OCTANE PANDA missing field first_activity_date.
[2023-03-31 19:23:55,723] DEBUG    processor/thread_3   Created adversary event for SHATTERED PANDA
[2023-03-31 19:23:55,737] DEBUG    processor/thread_4   Created adversary event for CYBORG SPIDER
[2023-03-31 19:23:55,739] DEBUG    processor/thread_5   Created adversary event for ALCHEMIST SPIDER
[2023-03-31 19:23:55,744] DEBUG    processor/thread_14  Created adversary event for CHRONO KITTEN
[2023-03-31 19:23:55,746] DEBUG    processor/thread_1   Created adversary event for FERAL SPIDER
[2023-03-31 19:23:55,747] DEBUG    processor/thread_8   Created adversary event for HIDDEN SPIDER
[2023-03-31 19:23:55,750] DEBUG    processor/thread_2   Created adversary event for SPECTRAL KITTEN
[2023-03-31 19:23:55,755] DEBUG    processor/thread_6   Created adversary event for COMPASS SPIDER
[2023-03-31 19:23:55,759] DEBUG    processor/thread_13  Created adversary event for SHINING SPIDER
[2023-03-31 19:23:55,763] DEBUG    processor/thread_0   Created adversary event for OCTANE PANDA
[2023-03-31 19:23:55,766] DEBUG    processor/thread_12  Created adversary event for SAMBA SPIDER
[2023-03-31 19:23:55,767] DEBUG    processor/thread_11  Created adversary event for HOLIDAY SPIDER
[2023-03-31 19:23:55,770] DEBUG    processor/thread_15  Created adversary event for HAYWIRE KITTEN
[2023-03-31 19:23:55,774] DEBUG    processor/thread_7   Created adversary event for OUTBREAK SPIDER
[2023-03-31 19:23:55,778] DEBUG    processor/thread_10  Created adversary event for SCATTERED SPIDER
[2023-03-31 19:23:55,786] DEBUG    processor/thread_9   Created adversary event for SLIPPY SPIDER
[2023-03-31 19:24:05,780] DEBUG    processor/thread_8   Created adversary event for COSMIC WOLF
[2023-03-31 19:24:07,406] DEBUG    processor/thread_10  Created adversary event for PARTISAN JACKAL
[2023-03-31 19:24:08,225] DEBUG    processor/thread_11  Created adversary event for LILY SPIDER
[2023-03-31 19:24:10,716] DEBUG    processor/thread_5   Created adversary event for VAPOR PANDA
[2023-03-31 19:24:10,726] DEBUG    processor/thread_12  Created adversary event for EMBER BEAR
[2023-03-31 19:24:10,827] DEBUG    processor/thread_9   Created adversary event for CHARIOT SPIDER
[2023-03-31 19:24:11,191] DEBUG    processor/thread_1   Created adversary event for HERMIT SPIDER
[2023-03-31 19:24:12,685] DEBUG    processor/thread_7   Created adversary event for WANDERING SPIDER
[2023-03-31 19:24:13,421] DEBUG    processor/thread_3   Created adversary event for VICE SPIDER
[2023-03-31 19:24:14,792] DEBUG    processor/thread_11  Created adversary event for ALPHA SPIDER
[2023-03-31 19:24:14,959] DEBUG    processor/thread_13  Created adversary event for RECESS SPIDER
[2023-03-31 19:24:16,752] DEBUG    processor/thread_15  Created adversary event for HIVE SPIDER
[2023-03-31 19:24:17,838] DEBUG    processor/thread_9   Created adversary event for DEADEYE HAWK
[2023-03-31 19:24:18,191] DEBUG    processor/thread_1   Created adversary event for GALACTIC OCELOT
[2023-03-31 19:24:20,358] DEBUG    processor/thread_6   Created adversary event for VERTIGO PANDA
[2023-03-31 19:24:21,323] DEBUG    processor/thread_0   Created adversary event for BLIND SPIDER
[2023-03-31 19:24:25,341] DEBUG    processor/thread_14  Created adversary event for JACKPOT PANDA
[2023-03-31 19:24:26,614] WARNING  pymisp/thread_10     The value of the attribute you're trying to add is an empty string, skipping it. Object relation: alias
[2023-03-31 19:24:26,617] DEBUG    processor/thread_10  Created adversary event for CHAOTIC SPIDER
[2023-03-31 19:24:27,480] DEBUG    processor/thread_2   Created adversary event for AQUATIC PANDA
[2023-03-31 19:24:27,694] DEBUG    processor/thread_3   Created adversary event for BRAIN SPIDER
[2023-03-31 19:24:27,751] DEBUG    processor/thread_7   Created adversary event for AVIATOR SPIDER
[2023-03-31 19:24:27,761] DEBUG    processor/thread_0   Created adversary event for FRINGE LEOPARD
[2023-03-31 19:24:27,781] DEBUG    processor/thread_4   Created adversary event for HAZY TIGER
[2023-03-31 19:24:27,828] DEBUG    processor/thread_13  Created adversary event for MIRAGE TIGER
[2023-03-31 19:24:27,851] DEBUG    processor/thread_15  Created adversary event for VETO SPIDER
[2023-03-31 19:24:27,879] WARNING  processor/thread_11  Adversary DEMON SPIDER missing field first_activity_date.
[2023-03-31 19:24:27,880] WARNING  processor/thread_11  Adversary DEMON SPIDER missing field last_activity_date.
[2023-03-31 19:24:27,880] DEBUG    processor/thread_11  Created adversary event for DEMON SPIDER
[2023-03-31 19:24:28,016] DEBUG    processor/thread_10  Created adversary event for BANISHED KITTEN
[2023-03-31 19:24:28,212] WARNING  processor/thread_11  Could not add or tag event ADV-203346 DEMON SPIDER (eCrime). Will retry in 0.3 seconds.
day is out of range for month: 0
[2023-03-31 19:24:28,331] DEBUG    processor/thread_7   Created adversary event for ROYAL SPIDER
[2023-03-31 19:24:28,384] DEBUG    processor/thread_5   Created adversary event for BITWISE SPIDER
[2023-03-31 19:24:28,633] DEBUG    processor/thread_3   Created adversary event for GOSSAMER BEAR
[2023-03-31 19:24:28,642] DEBUG    processor/thread_11  Created adversary event for REGAL JACKAL
[2023-03-31 19:24:28,698] WARNING  processor/thread_6   Adversary INTREPID JACKAL missing field first_activity_date.
[2023-03-31 19:24:28,698] WARNING  processor/thread_6   Adversary INTREPID JACKAL missing field last_activity_date.
[2023-03-31 19:24:28,699] DEBUG    processor/thread_6   Created adversary event for INTREPID JACKAL
[2023-03-31 19:24:28,820] DEBUG    processor/thread_12  Created adversary event for OUTRIDER TIGER
[2023-03-31 19:24:28,853] WARNING  processor/thread_9   Adversary VAMPIRE SPIDER missing field last_activity_date.
[2023-03-31 19:24:28,854] WARNING  pymisp/thread_9      The value of the attribute you're trying to add is an empty string, skipping it. Object relation: alias
[2023-03-31 19:24:28,863] DEBUG    processor/thread_9   Created adversary event for NEMESIS KITTEN
[2023-03-31 19:24:28,910] DEBUG    processor/thread_15  Created adversary event for SUNRISE PANDA
[2023-03-31 19:24:29,023] DEBUG    processor/thread_14  Created adversary event for ETHEREAL PANDA
[2023-03-31 19:24:30,586] WARNING  processor/thread_6   Could not add or tag event ADV-137920 INTREPID JACKAL (Hacktivist). Will retry in 0.3 seconds.
day is out of range for month: 0
Traceback (most recent call last):
  File "misp_import.py", line 377, in <module>
    main()
  File "misp_import.py", line 356, in main
    importer.import_from_crowdstrike(int(settings["CrowdStrike"]["init_reports_days_before"]),
  File "/home/rx118r/src/crowdstrike/MISP-tools-main/cs_misp_import/importer.py", line 303, in import_from_crowdstrike
    self.actors_importer.process_actors(actors_days_before, self.event_ids)
  File "/home/rx118r/src/crowdstrike/MISP-tools-main/cs_misp_import/actors.py", line 153, in process_actors
    if fut.result():
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/rx118r/src/crowdstrike/MISP-tools-main/cs_misp_import/actors.py", line 66, in batch_import_actors
    event: MISPEvent = self.create_event_from_actor(act, act_det)
  File "/home/rx118r/src/crowdstrike/MISP-tools-main/cs_misp_import/actors.py", line 372, in create_event_from_actor
    kao.add_tag(f"CrowdStrike:adversary:branch: {actor_branch}")
AttributeError: 'NoneType' object has no attribute 'add_tag'

Duplicates #111. Closing.


[Event Change Request] table 77 "Report Selections" Event OnGetEmailBodyCustomerTextOnAfterNotFindEmailBodyUsage

Describe the request
Can the above event have another boolean parameter added (called EmailBodyUsageFound)  and if the event subscriber sets this to true then the code does not exit...
if not FindEmailBodyUsageForCust(ReportUsage, CustNo, TempBodyReportSelections) then begin
    IsHandled := false;
    EmailBodyUsageFound := false;   // NEW CODE
    OnGetEmailBodyCustomerTextOnAfterNotFindEmailBodyUsage(
        ReportUsage.AsInteger(), RecordVariant, CustNo, TempBodyReportSelections, IsHandled, EmailBodyUsageFound);   // PARAMETER ADDED
    if not EmailBodyUsageFound then begin   // NEW CODE
        if IsHandled then
            exit(true);
        exit(false);
    end;   // NEW CODE
end;

The EmailBodyUsageFound needs to be a var parameter
Additional context
The reason for this is I want to use my own logic to populate the TempBodyReportSelections with the "Report ID" and the "Email Body Layout Code" of the report/layout I want to use for the email body.
The TempBodyReportSelections  is already a var parameter, but it can't be used at the moment because the code always exits and so the code further down that uses the TempBodyReportSelections never gets calledInternal work item: AB#493773

@JesperSchulz Just to give a bit more additional context... I can subscribe to OnSendEmailDirectlyOnBeforeSaveReportAsPDFInTempBlob to run my own logic to use an alternative report/layout for the attachment when sending via email, but I'm currently unable to do the same to the email body.  The requested change will allow me to use my own logic to use an alternative report/layout for the email body.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 14883.


[ie/elonet] Add support for Icareus-backed Elonet URLs

Description of your pull request and other information
This PR extends the Elonet extractor to support videos hosted on Icareus by synthesizing an Icareus URL and punting forward to the Icareus extractor. An example of such a video is https://elonet.finna.fi/Record/kavi.elonet_elokuva_107293.
I'm not sure how to pytest this (other than that, well, yt-dlp for that URL now works), as hatch test ElonetExtractor just gives me ERROR: not found: yt-dlp/test/test_download.py::TestDownload::test_ElonetExtractor.
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

If all Elonet videos are using Icareus now, we should just rewrite the extractor to always return url_results to Icareus, and we might as well use the Icareus metadata, since we're only using the opengraph title/description/thumb in ElonetIE anyways

If all Elonet videos are using Icareus now

I'm not 100% certain all are. Might be worth holding on to the fallback for a while?

might as well use the Icareus metadata

Mmm, Icareus IDs are not the same as Elonet's, if someone cares about that... 🤔

Metadata from the redirecting extractor (ElonetIE in this case) can override the metadata of the resolving extractor (IcareusIE in this case) if you return a url_transparent result.

Mmm, Icareus IDs are not the same as Elonet's, if someone cares about that...

However, the id is the one thing that can't be overridden. We would have to return _old_archive_ids in the url_transparent result info dict


add system tests to feature tests context include

Rails 5 now supports system/feature tests out the box, there type issystem this simply allows the feature contexts to be activated for system tests.

Thanks!
Released as 0.3.3

Perfect, thanks for merging and releasing so quickly 👍


Documentation Update: NamedTransformImage Servlet

Required Information

[x] AEM Version, including Service Packs, Cumulative Fix Packs, etc: ALL
[x] ACS AEM Commons Version: ALL
[x] Reproducible on Latest? yes

The documentation of the NamedImageTransformer should be extended. It should cover the fact, that the creation of renditions on-the-fly can be problematic, especially when the underlying image is large. In that case it can cause memory/garbage collection issues and even Out-Of-Memory situations, leading to overall slowdown of the instance up to a "crash" of the JVM.
Instead all required renditions should be pre-created using Processing profiles (AEM CS; AEM 6.x); in this case the rendition will only be created once, and for that case more optimized solutions are available (for example the use of imagemagick on AEM 6.x, and assetCompute on AEM CS), which are much more suited for that case.

@joerghoh This can be marked as not-compatible with AEM CS, especially now that we have web-optimized renditions (which is the proper way - better than using static renditions).
IIUC the on the fly renditioning is supported by AEM 6.5 since this is how the AEM Adaptive Servlet works.

Yeah, I think we should add the same warning also to the Core Components in case Renditions (of whatever size) are resized on the fly.

@joerghoh so you want to write up the docs PR?

https://github.com/Adobe-Consulting-Services/adobe-consulting-services.github.io/pull/248

I added a warning whenever this servlet is used. And even if this warning is turned off, we will see that it is turned off :-)
https://github.com/Adobe-Consulting-Services/acs-aem-commons/pull/3097


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/107408

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/107679

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/108645

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/109408

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/114233

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/116190

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/118058

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/119464


Offline Cache

We are building many microservices that might depend on Service Registry based Key Value Store. We originally thought of using either Consul from HashiCorp or use VSTS as the store since it has release variables.
However We are now thinking of using AppConfiguration for that purpose.,
Since this is going to be central configuration store, we do not want our services go kaboom when the AppConfiguration goes down for any odd reason.
I do see SetOfflineCache method. I am not sure if it is for this purpose of caching the values as a backup?
Are there any examples for SetOfflineCache?

Hi @kamalsivalingam, yes, the idea behind the offline cache is to save a last known good (LKG) copy of config that the application requested previously to the local environment of the user's application, so the application can still use LKG config when unexpected happens (for example, a network hiccup at the application start-up).
However, the users' application environments can be very different from one to another, so the solution of caching differs too. Each environment will need its implementation of IOfflineCache. Currently, the config provider provides offline file cache for applications hosted in Azure App Service. The caching file is encrypted and saved at $home/data/azureAppConfigCache (so the cache can be shared among multiple web workers). Here is a code snippet to enable it.
config.AddAzureAppConfiguration(option =>
{
    option.Connect(settings["ConnectionStrings:AppConfig"])
          .SetOfflineCache(new OfflineFileCache());
});

Please let us know if this is something you are looking for. Any feedback is welcome.

This is Awesome :..All the problems we wanted to solve using custom implementation is already taken care
:100:

does this also work locally? is %HOME% used there?

@vip32 when your app is not hosted in App Service, you can set your own offline file path via OfflineFileCacheOptions.Path. For example,
.SetOfflineCache(new OfflineFileCache(
    new OfflineFileCacheOptions() { Path = "your offline file path"}))

I amm getting the following error when using the above way.
System.Private.CoreLib: Exception while executing function: Function1. AzureKeyVault: The type initializer for 'AzureKeyVault.Function1' threw an exception. System.Text.Json: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.
P.S. I have created File "OfflineCache.json" before setOfflineCache() as it was throwing error for not finding the file

@vip32  Does this work for IIS hosted apps as well?  We still have apps hosted in IIS that have yet to move to the cloud we want them to consume remote configs as well.

Hello @pbhughes, @vishnu-97, @vip32, @kamalsivalingam,
we are evaluating how Offline Cache is used in different applications. Could you please help us understand the problem you are trying to solve by using Offline Cache?

@avanigupta I posted this question when I started exploring Azure AppConfig. The web apps and backend consumer apps hosted on Azure App Service that I worked on so far relies on Azure App Config and Azure KeyVault upon bootstrapping the application and so I didnt integrate the offline cache mechanism. However if you are using .net core Options pattern (IOptionsSnapshot) to auto refresh the configuration value in the app upon updating in app config and if you wanted a way to cache the copy of the configuration as a backup for azure app config, you could use this SetOfflineCache method to store the last known copy in a local store.

Hii Team,
We used them to serve as a backup incase azure is down for some reason.
Further, we also had the values in in memory, which get updated whenever
there was any update in AppConfiguration which provided faster access
reduced AppConfiguration requests.
So, offline cache, didn't actually came to be much use, it was jst there as
a backup of backup.
Thanks & regards,
Vishnu R Pillai
On Wed 14 Jul, 2021, 06:32 Avani Gupta, @.***> wrote:

Hello @pbhughes https://github.com/pbhughes, @vishnu-97
https://github.com/vishnu-97, @vip32 https://github.com/vip32,
@kamalsivalingam https://github.com/kamalsivalingam,
we are evaluating how Offline Cache is used in different applications.
Could you please help us understand the problem you are trying to solve by
using Offline Cache?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/AppConfiguration/issues/45#issuecomment-879502343,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMQJDJRQ6VC4QHIOE3PSSITTXTOY5ANCNFSM4G46INGA
.


6.9.13-SNAPSHOT contains a wierd version of XmlTest

Hi,
I was trying 6.9.13-SNASPHOT to test latest fixes and found something weird : the XmlTest class inside has Boolean getPreserveOrder() while repo here show it should rather be String getPreserveOrder.

I'm not sure to understand because the master branch has Boolean too: https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/xml/XmlTest.java#L712
  public Boolean getPreserveOrder() {
    if (m_preserveOrder == null) {
      return m_suite.getPreserveOrder();
    }


Avatar decorators are broken

Describe the bug
The preview and on the actual profile is broken.
Screenshots
If applicable, add screenshots to help explain your problem.

https://github.com/TakosThings/Fluent-Discord/wiki/Options#show-avatar-decorations


chromium sync

i read that chromium sync and api keys will be disabled how will that affect this

tbh i may not know what im talking about


Remove None and All default values for flag enums.

Continuation from https://github.com/crystal-lang/crystal/issues/7270#issuecomment-451947228
With most flag enums the semantics of None and All probably make sense. That's why they were added as defaults.
But they're not necessarily always semantically valid. When some of the flags are mutually exclusive, All is not possible. When setting at least one flag is required, None doesn't make sense either.
Because of this they better shouldn't be added to every flag enum.
Autogeneration can simply be removed. Where appropriate, these members can be added manually as None = 0 and All = ? (would it be Int32::MAX?).
Another option could be to configure the generation of these special enum members via the Flags annotation. For example, @[Flags(all: false)] would not autogenerate a None member.
We could keep the current semantics by default (plain @[Flags] annotation generates both special members) but allow to opt-out when the autogenerated values are not applicable. This is reasonable when most flag enums use All and None members (which I assume, but it's not based on evidence).
This solution adds more complexity to the compiler and another feature to the language. The alternative (remove + manual creation) is easier to follow because there is only one syntax and it's explicit.
Considering the highest desired state where each enum member would be documented with a individual description, autogeneration (in either form) is not useful because it can't generate meaningful documentation anyway.
Thus, I think I would prefer to remove autogeneration completely and require explicit declaration.
N.B. If we keep autogeneration in any form, there should be a fix to ensure autogenerated members are visible in the API docs. The documentation needs to be manually overridable.

@asterite are thinking about something like this?
enum Foo
  A
  B
  MyAll = all

  def self.all
    # generated with macros..
    # but how to detect that
    # MyAll should be skipped?
    A | B
  end
end

p Foo::MyAll

(see question comment)

@bew No, I'm thinking about this:
@[Flags]
enum Foo
  A
  B
  All = A | B
end

So if I have an enum with 20 fields, how would it look like from a user pov?
@[Flags]
enum Foo
  A
  B
  ...
  S
  T

  # Do I need to write it myself? (I hope not..)
  AtoT = A | B | ... | S | T
end

Yes, like that. How often is that needed? And how many times are you going to write that one All declaration? Just once. So it's not a big deal. It's simpler and the constants are visible, nothing is implicit.

Could it be a method, like Enum.all, or do we have to do create the method by ourselves?

Also: do you really need ALL that often that it must exist in the first place?

To me it's not about how often I need it, because I don't need flags very often. But I'm not representative of the users of this feature.
A simple use case would be to compute a mask of all possible values. Now I agree it's not common to use it as a enum member, but I think we should at least have a builtin method (on Enum) to get a All-like value.
Like:
struct Enum
  def self.all
     # logical OR of all members
  end
end

Then it's back to square one: the method name may conflict with the actual ALL.

How so? (what 'actual ALL' are you talking about?)

But I'm not representative of the users of this feature.

Who's a representative user of this feature? Please, reply here. If we don't get enough replies then I say we should remove that "feature".

Let's remove them. None is trivially specified manually.
If there's outcry I would vote to move the functionality to an auto-generated class method, something like SomeFlagEnum.all_flags (and then maybe SomFlagEnum.no_flags for parity). There's actually no good reason to have it as a member and this should avoid any conflicts with them.

There seem to be no strict objections against removing the default flags, so let's move forward with this.

Cool! I can work on that, it should be pretty easy to remove them. (but for 0.33.0, of course)

It turns out this can't be implemented in one go. We first need to remove the errors that happen if you try to redefine None and All in one release, and in a subsequent release redefine those while also removing their autogeneration.

I won't be able to work on this, but if someone wants to work on the first part, PRs are very welcome! 🙏

I am currently not on board with fully removing None and All. Obviously we can't remove them before 2.0 anyway. All actions that we do shouldn't assume that we'll eventually remove these and instead act in the best interest within the current limits.
I only support this PR for the moment:

#10497

Yeah, currently #10497 is the only piece tentatively agreed upon and up for approval. It's a preliminary step for eventual removal, but also useful on its own.
Any further action is due to discussion. And of course we can't do a breaking change before 2.0. There are however other short term options, for example configuration via a parameter on the Flag annotation.


php 7.2 compatibility

on PHP7.2 sizeof() throw an error when the param is not an array nor object:
sizeof(): Parameter must be an array or an object that implements Countable

@eavilesmejia AFAIK there is a major issue too and is that mcrypt was deprecated, how did you managed to make it work with PHP 7.2? Any advice? Thanks

Any news on this?

Going to take a look.

With #111. PHP Client will only works with PHP >=7.1.0.


Added SQLite, created initial migration, added some dependencies

Sorry, I meant to put this in my forked repo.


Improve colors for status

Status icons + Color
Finalize status icons
Add light and dark colors for Success, error, warning, and info
Documentation
Write out what breaking changes are
Create a visual change log and figure out where to put it
Create a prioritization framework
How often does it come up? How bad is it if we get it wrong? How hard is it to maintain?

Add christiahn to this when he has a public github

https://www.figma.com/file/a696svN2S7YNlZRYAkeLob/branch/oOxNcX78bLrB0wfE8M0oGl/Stellar-Design-System?type=design&node-id=9270-42567&mode=design&t=Nsrf4z2wyw0MYBWr-0

This is a clever and accessible approach to using shape and color to encode severity: https://twitter.com/JohnPhamous/status/1760444698857230360


Extraplanetary Launchpads: unfreeze and add spacedock link

is github just being unreliable right now...?

is github just being unreliable right now...?

Huh? 😕

https://github.com/KSP-CKAN/NetKAN/actions/runs/11407171543/job/31742788317?pr=10262#step:6:214

But what does that have to do with GitHub?

I missed that it was a metanetkan, and just went directly to the github releases

Yeah, looks like KerbalStats' download failure was being reported as KIS's, since it works with that recommendation removed.


Fixed pipelines options logic and jsdoc types

All fetch/sideload/write/read/undo/redo now follow these rules:

writePipeline should always run before saving the document to the history stack, should always affect what is saved to history.
readPipeline should always be executed after fetching a new copy of the document from history and before returning the document to the consumer, should never affect what is in history.

All tests pass. Although, maybe, we should add more tests to ensure compliance with these criteria.

I'll be writing some tests for these.


Inconsistencies in HARK.distribution: DiscreteDistributionLabeled, combine_indep_dstns, expected

There's something funky about how a few of our distribution operations work together.
The following code calls on several features in HARK.distribution:

DiscreteDistributionLabeled
combine_indep_dstns
expected

and breaks unexpectedly.
from HARK.distribution import expected, Normal
from HARK.model import discretized_shock_dstn
import numpy as np

shocks = {
    "gamma": Normal(mu= 0, sigma = 1)
}

approx = {
    "gamma" : {"N" : 7}
}

ds = discretized_shock_dstn(shocks, approx)

discretized_shock_dstn creates a DiscreteDistributionLabeled class for each shock passed in (as a dictionary), and then combines them. See here: https://github.com/econ-ark/HARK/blob/master/HARK/model.py#L44
Then the following code breaks:
def sample_f(shocks):
    return shocks["gamma"] * np.array([2,4])

expected(func = sample_f, dist = ds)

This produces an error:
Cell In[18], line 3, in sample_f(shocks)
      1 def sample_f(shocks):
----> 3     return shocks["gamma"] * np.array([2,4])

ValueError: operands could not be broadcast together with shapes (7,) (2,)

The issue being that the full grid of the discretized distribution is being passed (in a dictionary, 'labeled' form) to the function sample_f, instead of one value being passed in at a time. This doesn't typically raise an error, but it does when the function includes an array operation.
I believe these different functions in HARK.distribution were written by different people, so many it's a matter of harmonizing them.

Oh, this looks like it's because of the difference between expected and calc_expectations.
expected wants the function passed to it to take... the whole vector of probability atoms?
It says "This function should take the full array of distribution values
and return either arrays of arbitrary shape or scalars."
https://github.com/econ-ark/HARK/blob/master/HARK/distribution.py#L2430
That doesn't make sense to me... the values of either the probabilities or the realized values isn't enough to compute the expected value, and there isn't an argument for both.
I see this is widely used in the HARK models but the docs don't make sense to me. Can somebody explain?


How to set OpenAI key

how to set openAI key?

If you are using the sample Docker Compose configuration, you can access the local port bound to the backend-web-server service at localhost:9000 to log in to the management panel. Configure settings in the management panel.
Default superadmin account: admin
Default password: password
After logging in
In the Settings model, add a record with the name "openai_api_key" and the value as your API key.

thank you


[android] Fix typo in build script

Fix the invalid path for flatbuf and paho-mqtt-c tar.xz files

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #537. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


Fix: Fix multi-instances bug

Hello @rouftom ,
This PR fixes the bug that multiple instances cannot exist in a single page.
Can you merge this PR?
Thanks.

Hello @ShirasawaSama,
Can you provide more details regarding the bug you mentioned ?
Thanks.

When the page contains two instances, click the upload button for the second instance, but control the first instance.

@rouftom Because the id of each input instance is input-files.

@rouftom Could you release a new version to NPM?

Hello @ShirasawaSama,
Sorry I was busy the last few days, I'll post a new version tonight

Hello @ShirasawaSama,
I published a new release of the project


Implement WebShare API.

Implements WebShare API (for use with CHAPI wallet picker, etc).
(This PR is a re-base of PR #166)

PR moved to https://github.com/digitalcredentials/learner-credential-wallet/pull/244, closing this one.


Ft dev k8s cluster

WHAT DOES THIS PR DO?

Creates the dev k8s cluster infrastructure
Fixes the issues in ansible k8s creation role

WHAT ISSUES ARE RELATED TO THIS PR?

It adds the dev cluster

HOW DO I TEST OUT THIS PR?
WHICH ENDPOINTS SHOULD BE READY FOR TESTING?:
IS THERE ANY JENKINS CONSOLE LINK FOR CODE COVERAGE AND BUILD INFO?
ARE THERE ANY RELATED PRs?

Just thinking out loud: Is it possible to start thinking about IPv6 support in our IaC??

Thanks @danielogen.
I hadn’t thought about it yet about I definitely think we can take that direction moving forward.


Format response data for text/plain

Hello everyone,
I want to set the [Produces ("text/plain")] attribute for a method that returns an int value.
Unfortunately, the method returns a status code of 406.
Code:
        [Produces("text/plain")]
        public int Get()
        {
            return 1;
        }

Request:
without accept header
Response:
Error: response status is 406
In the documentation:

By default, ASP.NET Core supports the following media types:
application/json
text/json
text/plain

The documentation is not clear about this.
Do you know how to return an int value in text/plain format?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6ebcd752-af71-c483-eebb-0745bd04fb30
Version Independent ID: 1f678d5d-230b-0ad0-c925-15b13081f553
Content: Format response data in ASP.NET Core Web API
Content Source: aspnetcore/web-api/advanced/formatting.md
Product: aspnet-core
Technology: aspnetcore-webapi
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog manageable. If you believe there is a concern which hasn't been addressed, please file a new issue.


Bond futures

This work closes #428
This work closes #429

Added some tweaks to the code in a new commit.

Fine for me (except a small item in the data set).
At a later stage we will probably need to add some "basis" computation for bond futures. Also the computation of conversion factors may be useful but are not required at this stage.


Add interruptible to node and wf

TL;DR
Allow users to mark the WF/Node as interruptible so they can opt-in/out of spot usage. Spot machines are much cheaper than on-demand but have increased probability of interruption.
Type

[ ] Bug Fix
[x] Feature
[ ] Plugin

Are all requirements met?

[x] Code completed
[ ] Smoke tested
[x] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
The interruptible flag in CRD is plumbed through to task plugin where it take the appropriate action- containers task plugin may chose to run it on interruptible machines.
Tracking Issue
https://github.com/lyft/flyte/issues/207
Follow-up issue
NA

Codecov Report

Merging #86 into master will decrease coverage by 2.52%.
The diff coverage is 46.15%.

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
- Coverage    49.7%   47.17%   -2.53%
==========================================
  Files         123      109      -14
  Lines        7599     6584    -1015
==========================================
- Hits         3777     3106     -671
+ Misses       3455     3202     -253
+ Partials      367      276      -91

Impacted Files
Coverage Δ

pkg/apis/flyteworkflow/v1alpha1/iface.go
0% <ø> (ø)
:arrow_up:

pkg/controller/config/config.go
0% <ø> (ø)
:arrow_up:

pkg/apis/flyteworkflow/v1alpha1/nodes.go
4.83% <0%> (-0.17%)
:arrow_down:

...is/flyteworkflow/v1alpha1/zz_generated.deepcopy.go
0% <0%> (ø)
:arrow_up:

pkg/apis/flyteworkflow/v1alpha1/workflow.go
18.18% <0%> (-0.43%)
:arrow_down:

pkg/controller/nodes/executor.go
58.83% <100%> (+0.19%)
:arrow_up:

pkg/compiler/transformers/k8s/workflow.go
66.41% <100%> (+1.05%)
:arrow_up:

pkg/controller/nodes/node_exec_context.go
58.33% <38.46%> (-3.3%)
:arrow_down:

pkg/compiler/transformers/k8s/node.go
60% <40%> (-0.91%)
:arrow_down:

pkg/controller/nodes/task/plugin_state_manager.go

... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0411acd...12d487f. Read the comment docs.

Codecov Report

Merging #86 into master will decrease coverage by 0.03%.
The diff coverage is 53.7%.

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
- Coverage    49.7%   49.67%   -0.04%
==========================================
  Files         127      127
  Lines        7882     7918      +36
==========================================
+ Hits         3918     3933      +15
- Misses       3582     3601      +19
- Partials      382      384       +2

Impacted Files
Coverage Δ

pkg/apis/flyteworkflow/v1alpha1/iface.go
0% <ø> (ø)
:arrow_up:

pkg/controller/config/config.go
0% <ø> (ø)
:arrow_up:

pkg/apis/flyteworkflow/v1alpha1/nodes.go
4.83% <0%> (-0.17%)
:arrow_down:

...is/flyteworkflow/v1alpha1/zz_generated.deepcopy.go
0% <0%> (ø)
:arrow_up:

pkg/apis/flyteworkflow/v1alpha1/workflow.go
18.18% <0%> (-0.43%)
:arrow_down:

pkg/controller/nodes/executor.go
58.83% <100%> (+0.19%)
:arrow_up:

pkg/compiler/transformers/k8s/node.go
62.6% <100%> (+1.69%)
:arrow_up:

pkg/compiler/transformers/k8s/workflow.go
66.41% <100%> (+1.05%)
:arrow_up:

pkg/controller/config/config_flags.go
75.86% <100%> (+0.42%)
:arrow_up:

pkg/controller/nodes/node_exec_context.go
58.33% <38.46%> (-3.3%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a812e79...575bfd1. Read the comment docs.


[Question] How to supply Values

Hello, I'm using this node to control my AC, and for that I'm reading the temperatures with multiple xiomi temperature sensors that are zigbee. This devices only submit the temperature when the delta is 0.5º. This can take some time to make a report when heatig or cooling a room.
What is the best approach to have better results, submit the temperature periodically, like 30s, even if no changes is detected, or only when the sensor submits the change?
I'm getting a lot of Integral Locked, and I think that is related with a low Integral Time, I used your blog post to help calibrate the PID.
Thank You

I have it sending every 30 seconds! Probably it's to often! Can it be this the reason for having constant integral lock? How can I reset this without having to redeploy the flows?
Thank you

What is your setpoint and process value? Integral lock happens only, if both are very different.

Process value: 27.17
Set Point: 27
I don't think they are that far apart!
The values that I'm testing in the setting are:
PB - 0.07
Int Time: 360
Derv Time: 90

I know that my PB is a bit low, but I tested out with the method of doubling the overshoot, and I got that

My use case is heating, so it should not be that much difference:

process value 19.9
set point: 19.5
proportional band: 3
integral time: 21600
derivative time: 2700
output: 0
It shows me "Integral Locked", too. But this is fine as process is over set point and I don't want any heating.

How can you measure temperatures with an accuracy of 1/100 degree? I only found temperature sensors, with an accuracy of 1/10 degree but the deviance is given as +/-1°. So they measure something but not room temperature as it actually is. Consecutive sensor values sometimes differ +/-0.5°.

I think that I might have my values wrong!? Where in my setting did you get that my sensors are 1/100 of accuracy? To get the proportional band value I had the system running with only the set point and measured the bigger diference from my set point to the top of the curve, and I got 0.35, I doubled it and got the 0.7, and that was my starting point.
My sensors are zigbee from xiaomi, and they report at every 0.5º delta (or 1 hour since last report)
Then to get the integral time I doubled the time that was from the set point to the top of the curve, that was 180s, so, 360s... To my derivative I divided 180s for 2, so the 90s.

You said, your process value is 27.17. That is accuracy 1/100.
Regarding your proportional band: 0.7 means in your case with a set point of 27, that a process value of 27.7 will result in 1.0 output value. Is that, what you want?
Btw. how do you control your cooling process? How is an output value of 0.5 applied to the cooling process?

Give the PID loop data as often as you can, even if it has not changed.
To tune a loop it is absolutely essential to have a chart showing the process value and the output of the PID, so make a chart (if you have not got one already) and add the image here.  Then we will get an idea of whether it is reasonably tuned or not and whether you are sampling the temperature often enough.
You said the sensors only give you a value when the change is 0.5 degrees and asked what the best way to get better results is, well the answer to that is to use sensors with better performance.  I use DS18B20 which are very cheap, have a resolution of 1/16 C and can be polled every few seconds if necessary.  However the first thing to do is to get the best possible with what you have so post the chart and we will see what is going on.  You might have to scale the pid output to make it readable on the chart, it doesn't need to fill the chart, but we need to see the general trends.  You can use a Range node to scale the input range 0 to 1 to an output range that fits on the chart ok.
How are you driving the AC from the PID output?

My process value is the average of several sensor scattered around the room, I do this to get a more uniform temperature, I don't want a sensor that is closest to the ceiling to report a higher temperature and influence the cooling.
Regarding the output 1 at 27.7, probably I can make more like, output 1 when is 28.
To make the cooling I process the outpout in a range 0:1 to 1:0 and it inverts the values

Yes I have all the date logged in graphana, when I get home I will pull the data and paste the charts here! What do you need the Kp Ki and Kd variables or only the output to the AC?
To get the PID to the AC, I process the values from the PID and move them by a 0:1 - 0:5 range and that output is the speed that I send to my ac unit. So in the case of cooling, I use a 0:1 - 1:0 range and then 0:1 - 0:5 range. If its heating I will only use 0:1 - 0:5 range.
Thank you

The output value needs to be used accurately to control the AC. A value of 0.0 means: no AC at all, 0.5 means half of full AC power and 1.0 means maximum AC power. What you mean with the speed of an AC unit? If it has a speed control between off and 1 to 5, output value of 0.0 means off, while output of 0.5 should be translated to speed value 2.5 and output of 1.0 should be translated to speed value 5.

Using grafana (with Influx or similar presumably) is great, as you can always go back and see how it performed.  Make sure you keep a note of the tuning params too.  By the way, they aren't Kp, Ki and Kd in this algorithm they are PB, TI, and TD (proportional band, integral time and derivative time).  Yes I need to know those too please.
They aren't just different names for the same parameters, they are different constants in the maths.

I round it to integer, so 0 will mean that the ac will go to sleep mode, in this state e keeps using only the fans for a few minutes (to help mix the air) and the stop. Value of 1 to 5 is the speed of the fans, so 0.5 output will be 2.5, that will be the speed 3.

Using grafana (with Influx or similar presumably) is great, as you can always go back and see how it performed. Make sure you keep a note of the tuning params too. By the way, they aren't Kp, Ki and Kd in this algorithm they are PB, TI, and TD (proportional band, integral time and derivative time). Yes I need to know those too please.
They aren't just different names for the same parameters, they are different constants in the maths.

When I get home I'll pull the graph, since all is logged in influxdb I can go back in time and see what was the response of the PID to that settings.

The result of the PID working in the AC

The Values of the PID

I would take the derivative off, set it to zero, it won't do anything useful when you have such large steps in the apparent process values, all it does is add those spikes every time the process value steps, which aren't helpful and .  What does it look like on a much longer timescale?  Obviously with that resolution and the fact that you only have a few discrete speed settings you are never going to get really good control.

Main main goal is to keep the temperature always at the lowest speed, to ensure power savings! Yes I only have 5 speeds in the ac. I do this because the temperature regulation of the auto in my ac is very bad, and is almost always at full speed, making the power footprint very big. I want to ensure that the system reduces the speed when it does not need it to keep the temperature.

I am sure it can be made to work reasonably well, Set the Derivative Time to zero and show me how it does then, if possible over a few hours.
What have you got PB and TI set to currently?


Move isort settings into pyproject.toml

Description

Fixes isort vscode integration
Fixes an issue where tests was listed in src_paths but this was expected to be relative to python/morpheus

By Submitting this PR I confirm:

I am familiar with the Contributing Guidelines.
When the PR is ready for review, new or existing tests cover these changes.
When the PR is ready for review, the documentation is up to date with these changes.

/ok to test

/merge


Release gcp-metadata v0.7.0

This pull request was generated using releasetool.

/cc @JustinBeckwith @theacodes. releasetools seems to work fairly well. It would be nice if the destination of the Changelog was configurable. Instead of the CHANGELOG.md, I would strongly prefer it if a GitHub release was drafted with the content that is currently being put into CHANGELOG.md.

@ofrobots I believe releasetool tag will do exactly what you're saying :) . So you run releasetool start to create the PR, and after the PR lands you run releasetool tag to create the release on GitHub, and add the changelog for that release.

Sounds good. But I don't actually want a changelog in the repo.

I don't know, I've kinda come around to the idea. It's nice that the changelog is right there in PR.  As long as a tool maintains it, it doesn't bother me 🤷♂

We discussed this in the candidate repo & Justin and I agree on having
changelog.md. we can change it if absolutely necessary, but I do think
there are huge benefits to having the release notes as a reviewable part of
the PR.
On Tue, Jul 10, 2018, 4:36 PM Ali Ijaz Sheikh<EMAIL_ADDRESS>wrote:

Sounds good. But I don't actually want a changelog in the repo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stephenplusplus/gcp-metadata/pull/64#issuecomment-403998604,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUcxTQ07NYlr-wXJraiKDuLUwdSQysks5uFTpqgaJpZM4VKQvh
.

I don't like a file with the commit history and release notes (i.e. metadata) checked into a git repo.
It is not the end of the world though 🤷♂.
For the broadest reach, I would expect a tool to have sensible defaults, and if those don't match users' expectations, it would allow them tailor the tools' behavior. It looks great otherwise!

@theacodes I didn't see your comment until after I responded.

there are huge benefits to having the release notes as a reviewable part of the PR.

Yep, I agree with that. I don't think that a changelog committed to the repo is the best way to go about it though. I would expect a tool to generate release PRs like this: https://github.com/googleapis/nodejs-error-reporting/pull/159. It would be great if releasetools could support something like this, if not by default, then through some user specified flag.

The point of releasetool is to be consistent, so I don't foresee us adding
any user flags to change behavior.
On Tue, Jul 10, 2018, 8:04 PM Ali Ijaz Sheikh<EMAIL_ADDRESS>wrote:

Merged #64 https://github.com/stephenplusplus/gcp-metadata/pull/64.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stephenplusplus/gcp-metadata/pull/64#event-1726950200,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc7Fc-CaqZciqyZXN4pebKMuDbeYPks5uFWsjgaJpZM4VKQvh
.

If not a flag, what do you think of adding the details to the PR like @ofrobots is suggesting then?  I like it.

Aren't the details already in the PR in the changeset?
On Tue, Jul 10, 2018, 9:35 PM Justin Beckwith<EMAIL_ADDRESS>wrote:

If not a flag, what do you think of adding the details to the PR like
@ofrobots https://github.com/ofrobots is suggesting then? I like it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stephenplusplus/gcp-metadata/pull/64#issuecomment-404042446,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc1a8StiUlpU2_xEf7YzdrZcuClwDks5uFYCqgaJpZM4VKQvh
.


fix: remove madara as dep

Remove Madara as a dependency of the repo.
Time spent on this PR: 0.1 day
Resolves: #NA
Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[x] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Testing

What is the new behavior?
Madara is no longer a dependency of the conformance-test-utils crate.

Does this introduce a breaking change?

[ ] Yes
[x] No

Added changes to CHANGELOG.md

[x] Yes
[ ] No

nice job


add(plugin/deepgram): accepts optional parameter utterance_end_ms

Deepgram supports utterance-end which helps prevent the agent from interrupting the user https://developers.deepgram.com/docs/utterance-end
Adds support for this option in STTOptions and forwards the option in the live_config

Hey, shouldn't we listen to the EndUtterance event from Deepgram?

Logging out the event names, I didn't see an EndUtterance event. I'm not sure they provide this.

Logging out the event names, I didn't see an EndUtterance event. I'm not sure they provide this.

It is mentioned here, sorry it is UtteranceEnd not EndUtterance
https://developers.deepgram.com/docs/utterance-end

There is a similar PR here that listens to it https://github.com/livekit/agents/pull/737


Expose clippy-driver as an executable

We're working on moving cargo-clippy into cargo (https://github.com/rust-lang/rust-clippy/issues/3837). As a part of this, clippy-driver will need to be exposed by rustup.
Can clippy-driver be exposed into the bin folder as a shim? It's fine if it has a different name or something

We can add proxies sure, all we'll need is some indication of if it'll be a "normal" proxy or if you'll need something special.  If it's normal, then just give us the name you want and we can add it.
If you do that pretty soon, we might even be able to get it into 1.17.0 which we'd like to release soon (and will then be entering a larger refactoring effort which will make 1.18.0 a longer way off).

Just needs to be clippy-driver. Users should probably not invoke this themselves so feel free to give it another name, but just clippy-driver is fine.

See https://github.com/rust-lang/rustup.rs/pull/1679


Add new constructor to TestSever which allows configuring server instance before Build / Start is invoked.

This allows setting up Features for tests to match behavior of Kestrel and WebListener.
#967

@mattmazzola,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


Error Musume at start screen

Error musume is shown at start screen after 2-3 seconds even if I make no attempt at any action. Clearing cache does not work nor does attempting it with or without vpn. Kancolle still works in the internet explorer browser.

No idea. Try yuyuvn's build.


Programmatic Loading of Tests in External Jar

TestNG Version
6.14.3
Expected behavior
TestNG runs tests using added ClassLoader.
Actual behavior
TestNG not finding tests in ClassLoader.
Is the issue reproductible on runner?

[ ] Shell
[ ] Maven
[ ] Gradle
[ ] Ant
[ ] Eclipse
[ ] IntelliJ
[ ] NetBeans
[x] Java code

Test case sample
XmlSuite suite = new XmlSuite();
suite.setName("Invoked Run");

XmlTest test = new XmlTest();
test.setName("Invoked Run");
test.setParallel(XmlSuite.ParallelMode.NONE);
test.setXmlPackages(Collections.singletonList(new XmlPackage("com.tests.*")));
test.setTimeout(TimeUnit.SECONDS.toMillis(300));

TestNG testNg = new TestNG();
testNg.setOutputDirectory(persisPath);
testNg.setXmlSuites(Collections.singletonList(suite));

URLClassLoader customClassLoader = URLClassLoader.newInstance(jarUrls);
testNg.addClassLoader(customClassLoader);

testNg.run();
int status = testNg.getStatus();

When I run debug and hit the customClassLoader, I can run this successfully:
customClassLoader.loadClass("com.tests.SampleTest");

It retrieves the annotations appropriately as well.

ping @revof11 - Any updates on this ?

I have retried this using TestNG 7.0.0-beta5. I am not able to reproduce the issue.
I built a standalone jar which contains just 1 test class [ its a mavenised project wherein the test class resided under src/main/java ]
Here's how the test class looks like (this is what resides in the jar)
package com.rationaleemotions;

import org.testng.Reporter;
import org.testng.annotations.Test;

public class AppTest {

  @Test
  public void testMethod() {
    Reporter.log("Hello world", true);
  }

}

Here's the method
  private static void runTests(File jar) throws MalformedURLException {
    TestNG testNg = new TestNG();
    URLClassLoader customClassLoader = URLClassLoader.newInstance(new URL[]{jar.toURI().toURL()});
    testNg.addClassLoader(customClassLoader);
    XmlSuite suite = new XmlSuite();
    suite.setName("Invoked Run");

    XmlTest test = new XmlTest();
    test.setXmlSuite(suite);
    test.setName("Invoked Run");
    test.setParallel(XmlSuite.ParallelMode.NONE);
    XmlClass xmlClass = new XmlClass("com.rationaleemotions.AppTest");
    test.setClasses(Collections.singletonList(xmlClass));
    suite.setTests(Collections.singletonList(test));
    System.err.println(suite.toXml());

    testNg.setXmlSuites(Collections.singletonList(suite));

    testNg.setVerbose(2);
    testNg.run();
    int status = testNg.getStatus();
    System.err.println("Status ===> " + status);
  }

Output
Jar Location : /Users/krmahadevan/temp/issue_1999/target/issue_1999-1.0-SNAPSHOT.jar
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Invoked Run">
  <test thread-count="5" name="Invoked Run">
    <classes>
      <class name="com.rationaleemotions.AppTest"/>
    </classes>
  </test> <!-- Invoked Run -->
</suite> <!-- Invoked Run -->

Hello world
PASSED: testMethod

===============================================
    Invoked Run
    Tests run: 1, Failures: 0, Skips: 0
===============================================

===============================================
Invoked Run
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

Status ===> 0

Process finished with exit code 0

@krmahadevan Have you tried this by executing one jar into another using command line arguments ?
For me it is failing.!

I'm using 7.0.0
My use case is I wanted to create one executor service which can take up any jar and be able to execute any test present in 2nd jar. I have developed simple application and trying to execute the same. Please find github repo for demo and test repos
Demo - is executor service which triggers execution
test - contains sample test
https://github.com/akshaybadgujarjs/demo
https://github.com/akshaybadgujarjs/test
Please let me know in case any more information required.

The latest released version of TestNG is 7.6.1 (Needs JDK11). Can you please quickly give it a try using this and let me know if its still a problem? Based on your test result, I will try to dig in further and find out what is going on.

Sure will try and get back to you.!

Faced same issue with updated test ng version as well. Log text file attached for your reference.
logs.txt

@akshaybadgujarjs - Sounds like you are facing https://github.com/cbeust/testng/issues/2770
This has been fixed in 7.6.2-SNAPSHOT and should be available in the upcoming 7.6.2 released version.

My concern is not with java.nio.file.FileAlreadyExistsException: test-output/testng-reports.css this exception.
My query is that my test method is not getting executed at all.!
Pulsar Tests Suite
Total tests run: 0, Passes: 0, Failures: 0, Skips: 0

ah ok! Yeah. My bad. I got sidelined with that exception. I will take a look at this and let you know what is going on.

No problem. Thanks for your assistance.!

@akshaybadgujarjs - I can't reproduce the problem. The code works fine for me.
I cloned both your repositories and then built your test project.
After that I applied this patch

patch
diff --git a/build.gradle b/build.gradle
index 2905641..221d411 100644
--- a/build.gradle
+++ b/build.gradle
@@ -2,6 +2,7 @@ plugins {
 	id 'java'
 	id 'org.springframework.boot' version '2.4.13'
 	id 'io.spring.dependency-management' version '1.0.15.RELEASE'
+	id 'idea'
 }

 group = 'com.example'
@@ -21,8 +22,7 @@ dependencies {
 	implementation 'org.springframework.boot:spring-boot-starter-web'
 	testImplementation 'org.springframework.boot:spring-boot-starter-test'
 	// https://mvnrepository.com/artifact/org.testng/testng
-//     implementation group: 'org.testng', name: 'testng', version: '7.5'
-    implementation group: 'org.testng', name: 'testng', version: '7.0.0'
+    implementation group: 'org.testng', name: 'testng', version: '7.6.1'

     compileOnly "org.projectlombok:lombok:${lombokVersion}"
     annotationProcessor "org.projectlombok:lombok:${lombokVersion}"
@@ -36,5 +36,5 @@ tasks.named('test') {
 }

 bootJar {
-    mainClass = 'com.example.demo.DemoApplication'
-}
\ No newline at end of file
+    mainClass.set('com.example.demo.DemoApplication')
+}
diff --git a/src/main/java/com/example/demo/SampleController.java b/src/main/java/com/example/demo/SampleController.java
index b3d753b..7cd2788 100644
--- a/src/main/java/com/example/demo/SampleController.java
+++ b/src/main/java/com/example/demo/SampleController.java
@@ -1,5 +1,6 @@
 package com.example.demo;

+import java.net.MalformedURLException;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.web.bind.annotation.GetMapping;
 import org.springframework.web.bind.annotation.RestController;
@@ -7,11 +8,11 @@ import org.springframework.web.bind.annotation.RestController;
 @RestController
 public class SampleController {

-    @Autowired
-    private TestRunner testRunner;
+  @Autowired
+  private SimpleTestRunner simpleTestRunner;

-    @GetMapping("/execute-tests")
-    void executeTests(){
-        testRunner.triggerTestFromRequest();
-    }
+  @GetMapping("/execute-tests")
+  public void executeTests() throws MalformedURLException {
+    simpleTestRunner.runTestCase();
+  }
 }
diff --git a/src/main/java/com/example/demo/SimpleTestRunner.java b/src/main/java/com/example/demo/SimpleTestRunner.java
new file mode 100644
index 0000000..6938de0
--- /dev/null
+++ b/src/main/java/com/example/demo/SimpleTestRunner.java
@@ -0,0 +1,74 @@
+package com.example.demo;
+
+import java.io.File;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.util.Collections;
+import lombok.extern.slf4j.Slf4j;
+import org.springframework.beans.factory.annotation.Configurable;
+import org.springframework.stereotype.Component;
+import org.springframework.stereotype.Service;
+import org.testng.IInvokedMethod;
+import org.testng.IInvokedMethodListener;
+import org.testng.ITestResult;
+import org.testng.TestNG;
+import org.testng.xml.XmlClass;
+import org.testng.xml.XmlSuite;
+import org.testng.xml.XmlTest;
+
+@Slf4j
+@Service
+@Component
+@Configurable
+public class SimpleTestRunner {
+
+  private static final String JAR =
+      "/Users/krishnamahadevan/temp/testng-1999/test/build/libs/test-0.0.1-SNAPSHOT.jar";
+
+  public void runTestCase() throws MalformedURLException {
+    TestNG testNg = new TestNG();
+
+    URLClassLoader customClassLoader = URLClassLoader.newInstance(new URL[]{getURL()});
+    testNg.addClassLoader(customClassLoader);
+    XmlSuite suite = new XmlSuite();
+    suite.setName("Pulsar Tests Suite");
+
+    XmlTest test = new XmlTest();
+    test.setXmlSuite(suite);
+    test.setName("Pulsar Test");
+    test.setParallel(XmlSuite.ParallelMode.NONE);
+    XmlClass xmlClass = new XmlClass("com.example.test.PulsarTestsSample");
+    test.setClasses(Collections.singletonList(xmlClass));
+    test.setVerbose(2);
+    suite.setTests(Collections.singletonList(test));
+    log.info(suite.toXml());
+    testNg.setXmlSuites(Collections.singletonList(suite));
+
+    testNg.addListener(new LocalListener());
+    testNg.setVerbose(2);
+    testNg.run();
+    int status = testNg.getStatus();
+    log.info("Status ===> {} ", status);
+  }
+
+
+  private static URL getURL() {
+    try {
+      URL url = new File(JAR).toURI().toURL();
+      log.info("Working with URL {}", url);
+      return url;
+    } catch (MalformedURLException e) {
+      log.error("Could not load URL. Root cause: {}", e.getMessage(), e);
+      throw new RuntimeException(e);
+    }
+  }
+
+  public static class LocalListener implements IInvokedMethodListener {
+
+    @Override
+    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
+      log.info("About to run {}", method.getTestMethod().getQualifiedName());
+    }
+  }
+}

Now when I ran your main application and then triggered a get on http://localhost:8080/execute-tests, I noticed that the test runs perfectly fine.
I have attached the logs.
logs.txt

I will give it a try and confirm.

Hello @krmahadevan
I checked it. It is working fine in IDE. I suppose you have executed it from IDE as well. Is that correct ? Because from IDE even I didn't faced any issue.
The main problem occurs when I try to execute it from command line. If you can reproduce please let me know.!
FYI, I'm using given command to create build and run it
==> To Build
./gradlew clean build -x test
==> To Run
java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -jar -Dspring.profiles.active=local /Users/akshay.badgujar_js/IdeaProjects/demo/build/libs/demo-0.0.1-SNAPSHOT.jar
commnad line logs.txt

I checked it. It is working fine in IDE. I suppose you have executed it from IDE as well. Is that correct ? Because from IDE even I didn't faced any issue.

I didnt see you mention anywhere on the mode of running it. So I ran it from within the IDE.
I was able to get the CLI running by using
java -cp build/libs/demo-0.0.1-SNAPSHOT.jar -Dloader.path=file:/Users/krishnamahadevan/temp/testng-1999/test/build/libs/ org.springframework.boot.loader.PropertiesLauncher

Refer here

@krmahadevan Have you tried this by executing one jar into another using command line arguments ? For me it is failing.!

Actually it was my first question about running it via command line.
Is your test executed after running it via CLI ?
For me it is still status 8 even after using your method for jar execution.
Running jar is not an issue. Once I calls for API, tests does not get executed. It gives me status 8 instead of 0 thats the concern.
Logs attached for your reference.
CLI Run with status 8.txt

@akshaybadgujarjs - It works for me
This is the command that I am using:
java -cp build/libs/demo-0.0.1-SNAPSHOT.jar -Dloader.path=file:/Users/krishnamahadevan/temp/testng-1999/test/build/libs/ org.springframework.boot.loader.PropertiesLauncher

2022-12-01 11:55:31.946  INFO 18916 --- [nio-8080-exec-1] com.example.demo.SimpleTestRunner        : Working with URL file:/Users/krishnamahadevan/temp/testng-1999/test/build/libs/test-0.0.1-SNAPSHOT.jar
2022-12-01 11:55:31.966  INFO 18916 --- [nio-8080-exec-1] com.example.demo.SimpleTestRunner        : <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Pulsar Tests Suite">
  <test thread-count="5" name="Pulsar Test" verbose="2">
    <classes>
      <class name="com.example.test.PulsarTestsSample"/>
    </classes>
  </test> <!-- Pulsar Test -->
</suite> <!-- Pulsar Tests Suite -->

2022-12-01 11:55:32.025  INFO 18916 --- [nio-8080-exec-1] com.example.test.PulsarTestsSample       : PulsarTestsSample constructor
2022-12-01 11:55:32.079  INFO 18916 --- [nio-8080-exec-1] com.example.demo.SimpleTestRunner        : About to run com.example.test.PulsarTestsSample.sampleTest
2022-12-01 11:55:32.082  INFO 18916 --- [nio-8080-exec-1] com.example.test.PulsarTestsSample       : sampleTest
PASSED: sampleTest
        Add Restaurant Slot With No Holiday Slot And Verify

===============================================
    Pulsar Test
    Tests run: 1, Failures: 0, Skips: 0
===============================================

===============================================
Pulsar Tests Suite
Total tests run: 1, Passes: 1, Failures: 0, Skips: 0
===============================================

2022-12-01 11:55:32.181  INFO 18916 --- [nio-8080-exec-1] com.example.demo.SimpleTestRunner        : Status ===> 0

Okay thank you @krmahadevan for all the assistance and help here🙂
Will check and confirm.!

It worked as expected. Thank you so much.! 🙂


task_for_pid-allow permission

Is it possible to add the task_for_pid-allow permission to the Xcode ktrw_kext_loader project so that to be able to debug this app?
I am trying to run this project on iPhone10, 4 (iOS 13.2.2) which has missed kernel parameters and debugging capabilities would be very useful.
I use the Checkra1n JB tool which provides us with tfp0 functionality. tfp0 requires task_for_pid-allow permission.
What would you recommend in this case?
Thanks.

I’m actively working on adding support for checkra1n and iOS 13.2.2.

@bazad ah good to know! i started to port this to 13.2 as well, got to the point that the kext is loaded, but it panics as soon as it gets started for "Kernel instruction fetch abort".

After examining a few different approaches, I feel that the best solution under the current checkra1n feature set is to provide ktrw_kext_loader as a command-line utility (which can be easily signed with task_for_pid-allow) instead of an app.

@bazad Thanks. Originally, I also thought about making the kext loader as a command line tool.
To add new platform, iPhone 10.4 in my case, should we add new <hardware-version>_<build-version>.txt in the ktrw_kext_loader/kernel_symbols folder and do changes in the platform initialization platform?
Or using it on iOS 13.2.x requires more changes?

Hi, @NewDwarf
I'm trying to making the kext loader CLI tools. My environ is iPhone 10,4(13.2.2) similar to you.
Could you get the KASLR slide successfully?
I use Checkr1an and  I can do tfp0 but failed to get task_info.
The code snip is below.
kern_return_t ret;
uint64_t kbase, kslide;
mach_msg_type_number_t cnt = TASK_DYLD_INFO_COUNT;
task_dyld_info_data_t dyld_info;

ret = init_tfp0();
...
INFO("got mach port = 0x%x", tfp0);
ret = task_info(tfp0, TASK_DYLD_INFO, (task_info_t)&dyld_info, &cnt);
if(ret != KERN_SUCCESS) {
    ERROR("could not get slide...");
    return -1;
}
kslide = dyld_info.all_image_info_size;
kbase = kslide + 0xFFFFFFF007004000;
INFO("kbase: 0x%llx" , kbase);
INFO("kslide: 0x%llx" , kslide);
...

and result is below
iPhone:/usr/ktrw root# ldid -e ktrw
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.springboard.debugapplications</key>
        <true/>
        <key>get-task-allow</key>
        <true/>
        <key>proc_info-allow</key>
        <true/>
        <key>task_for_pid-allow</key>
        <true/>
        <key>run-unsigned-code</key>
        <true/>
        <key>platform-application</key>
        <true/>
    </dict>
</plist>
iPhone:/usr/ktrw root# ./ktrw
[+] got mach port = 0x903
[-] could not get slide...

Is there another way to get kslide with tfp0?
Thanks.

I've just pushed a change that should bring partial support to iPhone10,1 17B102 (iOS 13.2.2). ktrw_kext_loader runs fine as a command-line tool, but there appear to be stability problems when the ktrw_gdb_stub kext is running on iOS 13 which causes the phone to eventually panic. I'm looking into this and I'll update this thread with further info.

It looks like the issue is actually in the ktrw_kext_loader's KTRR bypass, not the kext. I think the problem is due to changes in iOS 13's io_map() implementation causing the mappings allocated by the kext loader to overlap with other data. I'm working on a fresh implementation that avoids the need for io_map() entirely.

@cdpython The same problem and, I guess, it is described here
@bazad suggested to use init_with_unsafe_heap_scan() in his last commit.
Many things are ready and I am going to collect the kernel addresses for linking and memory management related addresses.

@bazad Could you also add support of the iPhone 10,4 platform, please?
The same build (17B102) is used for both 10,1 and 10,4 platforms.
I did internal changes to verify that it works:
./ktrw_kext_loader ktrw_gdb_stub.ikext [+] Platform: iPhone10,4 17B102 [+] task_for_pid(0) = 0xa07 [!] Could not find the kernel base address [!] Trying to find the kernel base address using an unsafe heap scan! [+] KASLR slide is 0x1217c000 [+] Kext ktrw_gdb_stub.ikext loaded at address 0xffffffe0dd14c000

@NewDwarf Will do. Just to confirm, did you use the same offsets/addresses as iphone10,1 17B102, or did you have to change any of them? I don’t have an iphone10,4 to verify for myself.

@bazad Thanks. Yes, I used exactly the same offsets/addresses as iphone10,1 17B102.
Here is the patch
diff --git a/ktrw_kext_loader/kernel/kernel_parameters.c b/ktrw_kext_loader/kernel/kernel_parameters.c
index 84af7db..a696ec5 100644
--- a/ktrw_kext_loader/kernel/kernel_parameters.c
+++ b/ktrw_kext_loader/kernel/kernel_parameters.c
@@ -61,6 +61,7 @@ static struct platform_initialization offsets[] = {
 	{ "iPhone10,6", "16E227", offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "16G77",  offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "17B102", offsets__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", offsets__iphone10_1__17B102 },
 };

 // ---- Address initialization --------------------------------------------------------------------
@@ -90,6 +91,7 @@ static struct platform_initialization addresses[] = {
 	{ "iPhone10,6", "16E227", addresses__iphone10_6__16E227 },
 	{ "iPhone10,1", "16G77",  addresses__iphone10_1__16G77  },
 	{ "iPhone10,1", "17B102", addresses__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", addresses__iphone10_1__17B102 },
 };

 // ---- Public API --------------------------------------------------------------------------------
diff --git a/ktrw_kext_loader/kernel_call/kernel_call_parameters.c b/ktrw_kext_loader/kernel_call/kernel_call_parameters.c
index cccad66..bd174c5 100644
--- a/ktrw_kext_loader/kernel_call/kernel_call_parameters.c
+++ b/ktrw_kext_loader/kernel_call/kernel_call_parameters.c
@@ -61,6 +61,7 @@ static struct platform_initialization offsets[] = {
 	{ "iPhone10,6", "16E227", offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "16G77",  offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "17B102", offsets__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", offsets__iphone10_1__17B102 },
 };

 // ---- Address initialization --------------------------------------------------------------------
@@ -99,6 +100,7 @@ static struct platform_initialization addresses[] = {
 	{ "iPhone10,6", "16E227", addresses__iphone10_6__16E227 },
 	{ "iPhone10,1", "16G77",  addresses__iphone10_1__16G77  },
 	{ "iPhone10,1", "17B102", addresses__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", addresses__iphone10_1__17B102 },
 };

 // ---- Public API --------------------------------------------------------------------------------
diff --git a/ktrw_kext_loader/ktrr/ktrr_bypass_parameters.c b/ktrw_kext_loader/ktrr/ktrr_bypass_parameters.c
index be70700..6ae3b07 100644
--- a/ktrw_kext_loader/ktrr/ktrr_bypass_parameters.c
+++ b/ktrw_kext_loader/ktrr/ktrr_bypass_parameters.c
@@ -52,6 +52,7 @@ static struct platform_initialization offsets[] = {
 	{ "iPhone10,6", "16E227", offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "16G77",  offsets__iphone10_1__16C101 },
 	{ "iPhone10,1", "17B102", offsets__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", offsets__iphone10_1__17B102 },
 };

 // ---- KTRR parameter initialization -------------------------------------------------------------
@@ -131,6 +132,7 @@ static struct platform_initialization parameters[] = {
 	{ "iPhone10,6", "16E227", parameters__iphone10_6__16E227 },
 	{ "iPhone10,1", "16G77",  parameters__iphone10_1__16G77  },
 	{ "iPhone10,1", "17B102", parameters__iphone10_1__17B102 },
+	{ "iPhone10,4", "17B102", parameters__iphone10_1__17B102 },
 };

 // ---- Public API --------------------------------------------------------------------------------

After much investigation it appears that there are stability issues that occur when changing TTBR1_EL1 to remap the kernel. I haven't been able to pin down exactly what's causing these issues. However, given that checkra1n is likely to support loading kexts in the future anyway, remapping the kernel will become obsolete.
Unfortunately, this means that KTRW is stuck on 12.4 until checkra1n is updated with a kext loading framework.

@bazad
I guess on executing ktrr_bypass.c:309
...
else if (insn == 0xd5182020) { // MSR TTBR1_EL1, X0
    DEBUG_TRACE(1, "Hijacking TTBR1_EL1 %llx -> %llx", read_X(0), ttbr1_el1);
    write_X(0, ttbr1_el1);
...

Yes, that's one part of it; io_map() was also problematic.

@bazad
The full log of loading the kext.
[+] Platform: iPhone10,4 17B102
[+] task_for_pid(0) = 0x1407
[!] Could not find the kernel base address
[!] Trying to find the kernel base address using an unsafe heap scan!
[D] Found kernel pointer 0xfffffff026e47030
[+] KASLR slide is 0x1f538000
[D] Allocated kernel buffer at 0xffffffe035464000
[D] Set CPU 0 to halt on next reset
[D] Halted CPU 0 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 0
[D] Set CPU 1 to halt on next reset
[D] Halted CPU 1 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 1
[D] Set CPU 2 to halt on next reset
[D] Halted CPU 2 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 2
[D] Set CPU 3 to halt on next reset
[D] Halted CPU 3 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 3
[D] Set CPU 4 to halt on next reset
[D] Halted CPU 4 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 4
[D] Set CPU 5 to halt on next reset
[D] Halted CPU 5 in debug state
[D] Skipping KTRR initialization
[D] Hijacking TTBR1_EL1 809608000 -> 8269b4000
[D] Hijacking TTBR1_EL1 806e68000 -> 8269b4000
[D] Restarting CPU 5
[D] Loading kext ktrw_gdb_stub.ikext
[D] Resolved _IOSleep = 0xfffffff02762d68c
[D] Resolved __disable_preemption = 0xfffffff0272086e4
[D] Resolved __mh_execute_header = 0xfffffff02653c000
[D] Resolved _const_boot_args = 0xfffffff026e3e870
[D] Resolved _kernel_map = 0xfffffff026e3d658
[D] Resolved _kernel_memory_allocate = 0xfffffff027188af4
[D] Resolved _kernel_thread_start = 0xfffffff027132c54
[D] Resolved _ml_nofault_copy = 0xfffffff02720ecf8
[D] Resolved _thread_deallocate = 0xfffffff0271313a0
[D] Resolved _vsnprintf = 0xfffffff02751e0e4
[D] kext = ffffffe0ca230000
[D] Protecting segment 0-4000 as 1
[D] Protecting segment 4000-10000 as 5
[D] Protecting segment 10000-2c000 as 3
[D] Protecting segment 2c000-30000 as 3
[D] Protecting segment 30000-34000 as 1
[D] Starting kext ktrw_gdb_stub.ikext
[D] _kext_start returned 0x0
[+] Kext ktrw_gdb_stub.ikext loaded at address 0xffffffe0ca230000```

I didn't get the kernel panic but lldb is not able to connect to the gdb-server.
```lldb kernelcache.release.iphone10_decomp
(lldb) target create "kernelcache.release.iphone10_decomp"
Current executable set to 'kernelcache.release.iphone10_decomp' (arm64).
(lldb) settings set plugin.dynamic-loader.darwin-kernel.load-kexts false
(lldb) gdb-remote 39399
error: failed to get reply to handshake packet

./ktrw_usb_proxy 39399
Found KTRW device c03
Opened KTRW device

 <- +
 <- $QStartNoAckMode#b0
EOF socket

@NewDwarf Yeah, that does not look inconsistent with what I've seen. The corruption sometimes manifests as a hang.
It does look like disabling KTRR for both the MMU and AMCC through kernel __TEXT_EXEC patches via checkra1n would solve the issue fully. In addition to fixing the stability issues, this would remove the need for a KTRR bypass in ktrw_kext_loader itself and make kext loading much faster. I'm thus going to postpone any iOS 13 workarounds until this feature becomes available in checkra1n.

@NewDwarf I just realized that the behavior you saw might have actually been due to a bug in the USB stack that broke connectivity on macOS 10.15. That bug was fixed recently, but the instability on iOS 13 remains and is still pending on updates to checkra1n.

@bazad Thanks for the update. That's is good news!
...but recently I updated iPhone 8 to iOS 13.3.
I afraid all kernel symbol offsets as well as struct offsets in the kernel_call_parameters.c are changed and it is necessary to fix them.
Do you have updated symbol offsets for iOS 13.3?

@bazad
I fixed offsets for iOS 13.3 (17C54). iphone10_1 and iphone10_4 have same offsets.
Could you add them to the code base to support iOS 13.3, please?
`static void
addresses__iphone10_1__17C54() {
STATIC_ADDRESS(allproc) = 0xFFFFFFF0091EEC30;
}
static void
parameters__iphone10_1__17C54() {
gPhysBase                   = kernel_read64(SLIDE(0xFFFFFFF00790AB68));
gVirtBase                   = kernel_read64(SLIDE(0xFFFFFFF00790AB70));
rorgn_begin                 = kernel_read64(SLIDE(0xFFFFFFF00790AE58));
rorgn_end                   = kernel_read64(SLIDE(0xFFFFFFF00790AE60));
cpu_ttep                    = kernel_read64(SLIDE(0xFFFFFFF00790A7C0));
kernel_pmap                 = kernel_read64(SLIDE(0xFFFFFFF00790A7A0));
ADDRESS(pmap_find_phys)     = SLIDE(0xFFFFFFF007CCBDAC);
ADDRESS(ml_phys_read_data)  = SLIDE(0xFFFFFFF007CD83EC);
ADDRESS(ml_phys_write_data) = SLIDE(0xFFFFFFF007CD8690);
ADDRESS(ml_io_map)          = SLIDE(0xFFFFFFF007CDDD44);
ADDRESS(ldr_w0_x0__ret)     = SLIDE(0xFFFFFFF007BDAF34);
ADDRESS(str_w1_x0__ret)     = SLIDE(0xFFFFFFF007BDAFB8);
ADDRESS(CpuDataEntries)     = SLIDE(0xFFFFFFF0091D2C98);
}
static void
addresses__iphone10_1__17C54() {
ADDRESS(IOUserClient__vtable)                = SLIDE(0xFFFFFFF007920CD8);
ADDRESS(IORegistryEntry__getRegistryEntryID) = SLIDE(0xFFFFFFF00810696C);
}
iPhone10,4_17C54.txt
__disable_preemption        0xFFFFFFF007CD8BC4
__enable_preemption         0xFFFFFFF007CD8BF0
__mh_execute_header         0xFFFFFFF007004000
_const_boot_args            0xFFFFFFF00790A870
_IOSleep                    0xFFFFFFF0080FC218
_kernel_map                 0xFFFFFFF007909658
_kernel_memory_allocate     0xFFFFFFF007C58B6C
_kernel_thread_start        0xFFFFFFF007C02C8C
_ml_nofault_copy            0xFFFFFFF007CDF1D8
_panic                      0xFFFFFFF00903A9E4
_paniclog_append_noflush    0xFFFFFFF007BDD6D0
_thread_deallocate          0xFFFFFFF007C013D8
_vsnprintf                  0xFFFFFFF007FECB98
Additionally I verified that offsets in the ipc_entry, ipc_space, proc, task structs were not changed.static void
offsets__iphone10_1__17B102() {
kernel_slide_step                       = 0x4000;
SIZE(ipc_entry)                         = 0x18;
OFFSET(ipc_entry, ie_object)            = 0;
OFFSET(ipc_space, is_table_size)        = 0x14;
OFFSET(ipc_space, is_table)             = 0x20;
OFFSET(proc, p_list_next)               = 0;
OFFSET(proc, task)                      = 0x10;
OFFSET(proc, p_pid)                     = 0x68;
OFFSET(task, itk_space)                 = 0x320;
OFFSET(task, bsd_info)                  = 0x380;
STATIC_ADDRESS(kernel_base)             = 0xFFFFFFF007004000;
}can be used asstatic void
offsets__iphone10_1__17C54() {
kernel_slide_step                       = 0x4000;
SIZE(ipc_entry)                         = 0x18;
OFFSET(ipc_entry, ie_object)            = 0;
OFFSET(ipc_space, is_table_size)        = 0x14;
OFFSET(ipc_space, is_table)             = 0x20;
OFFSET(proc, p_list_next)               = 0;
OFFSET(proc, task)                      = 0x10;
OFFSET(proc, p_pid)                     = 0x68;
OFFSET(task, itk_space)                 = 0x320;
OFFSET(task, bsd_info)                  = 0x380;
STATIC_ADDRESS(kernel_base)             = 0xFFFFFFF007004000;
}`
But I didn't verify
`static void
offsets__iphone10_1__17B102() {
OFFSET(ipc_port, ip_kobject)                             = 104;
OFFSET(proc, p_ucred)                                    = 0x100;
OFFSET(task, bsd_info)                                   = 0x380;
SIZE(IOExternalTrap)                                     = 24;
OFFSET(IOExternalTrap, object)                           = 0;
OFFSET(IOExternalTrap, function)                         = 8;
OFFSET(IOExternalTrap, offset)                           = 16;
OFFSET(IORegistryEntry, reserved)                        = 16;
OFFSET(IORegistryEntry__ExpansionData, fRegistryEntryID) = 8;
VTABLE_INDEX(IOUserClient, getExternalTrapForIndex)      = 0x5c0 / 8;
VTABLE_INDEX(IOUserClient, getTargetAndTrapForIndex)     = 0x5c8 / 8;
}
static void
offsets__iphone10_1__17B102() {
SIZE(cpu_data_entry)                   = 16;
OFFSET(cpu_data_entry, cpu_data_vaddr) = 8;
OFFSET(cpu_data, cpu_regmap_paddr)     = 63 * 8;
OFFSET(cpu_data, ed_mmio)              = 59 * 8;
OFFSET(cpu_data, utt_mmio)             = 62 * 8;
}`

@bazad
Above offsets and addresses work fine on iOS 13.3. I got the kernel debugger functionality. It works pretty stable. At least, I didn't face with kernel panics.
Thanks for the great project!
...closing this issue.


Change ViewSets to actual view sets

Minor change, eliminates a spurious use of multisets, seems to work.
Also removes View: nothing uses it.

Would it be worth merging the different view types (OView/DView) into a single View type? I'm assuming that was the original intention of View.

@BenSimner Not sure how you would—some are Lists, some are Multisets.  What would be nice is changing the names to be more descriptive.  For example, DView is really a ViewSignature, and OView could be OrderedView.

What is the point of an ordered view here then?

@BenSimner I think ordered views were required to make the matching logic in reification happen, but I can't remember.  When @mjp41 returns we can ask him?  (I think it was his change, and I forget the rationale).

Incidentally, it might also be worth either renaming GView to GuardedView, or deleting it in favour of eg. View<GuardedFunc<'var>>.  These types might be getting a bit out of hand at this point though.  I think it'd be worthwhile to make it clearer that GView puts the guard on the func, not the view itself.

We need a bonfire of the type aliases soon.

Soon

@septract Is this ready to merge?


Automate selection exercise and dummy applicant data on staging

To do
Create a bank of testing data, from production. 3,000 candidates that will have some of their information obscured, for security (phone number and email address to contain random characters at the end i.e. jeremy123@judicial)
Hold this data in the db to be called upon when needed to fill in an exercise with relevant/real information.
All our test data will have the same login password, but unique email addresses. This will allow us to login as any user on Apply for testing purposes.
Acceptance criteria
Generate a bank of test data

[x] 3,000 records from prod to be stored here xxx

[x] Change applicants phone numbers to 08000 000 000

[x] Add sequential numbers to the end of their email address before the @ i.e. Justin123@, Barbara124@, Lawrence125

[x] Generate the same password for each applicant (send this to developers in Slack)

Load test application data

[x] When an exercise is published and approved a Create test applications button will appear

[x] When this button is selected a pop up appears to select applicant number. Chose between 1-3,000 then test applications will be generated from xxx

[x] Once successful the Success! button will appear and then move to Process late applications. (this is the usual flow)

Design
Create test applications button

Choose application number

Success state

Applications table

We need to go a lot further with cleaning the dummy data. Plus please ensure it is not publicly accessible!
Would suggest at least the following additional steps however please use your own diligence to ensure that no personal or identifiable data is used:

[ ] Replace the email addresses with our own dummy ones (rather than appending numbers to existing addresses). e.g. could use Shark Lasers / Mailinator or another dummy email generator (if we need to access emails) or simply a made up domain like @jac-dummy-email.jac
[ ] Ensure Assessors data is replaced with dummy data
[ ] Look to replace character information details with 'lorem ipsum' or similar. In some instances the explanations include fairly sensitive data
[ ] Look to replace qualification place names


Use build tools infra for kustomize and controller-gen

Instead of manually downloading kustomize and controller-gen every
time the Dapper image needs to be rebuilt, declare them as build tools
dependencies, so that they are vendored in. Build steps which need the
tools then ensure they are built.
This results in faster builds when the images need to be rebuilt
locally (many of the vendored dependencies are shared with the
operator and would be downloaded anyway).
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

curl: (18) transfer closed with 69405816 bytes remaining to read
The command '/bin/sh -c curl -Lo /usr/bin/operator-sdk "https://github.com/operator-framework/operator-sdk/releases/download/v${OPERATOR_SDK_VERSION}/operator-sdk-v${OPERATOR_SDK_VERSION}-x86_64-linux-gnu" &&     chmod a+x /usr/bin/operator-sdk' returned a non-zero code: 18
time="2021-04-09T15:11:26Z" level=fatal msg="exit status 18"


Add support for percentage based dimensions

percentage based dimensions for? The nf-graph container itself?

Yeah for the nf-graph component, instead of pixel based width/height

So this is tricky.
The pixel values passed to width and height are used to calculate the scale of the graphed elements. In order to get those, I'd need to check to see if what was passed happened to be a string ending with "%", then setup an event to detect when the SVG has resized. ... that's where this gets tricky. The only reliable way I know of to get when the SVG has been resized is to poll repeatedly and check it's computed size. This is because while a document resize event will help, it's not the only way to resize the parents of the SVG of the nf-graph component. If any DOM parent of the SVG element is updated, it will resize the SVG, and we need to know. Unfortunately there isn't a great event for that. MutationObservers may help, but aren't reliable in every browser, and may have performance implications... So I'd want to set them up only when the size was set to a percentage, and tear them down otherwise.

I'm not saying we're definitely not going to implement this. In fact, it might even end up being a requirement for the project this was birthed from. I'm really saying that this is a non-trivial change that could end up being buggy or slow on some platforms.

I'm adding wontfix to this and closing it, for now.
BUT I will keep this in mind because it's likely to come up on our app here.

@blesh I figured that would be case. Basically all the work would have to be invoked after the node is inserted in the DOM. Thanks anyway!

This could be the solution: EmberSherpa/ember-element-resize-detector

@ming-codes as I recall, we used something like that to handle resizing these components at Netflix


Helper text for new tag

Hi,
Does anyone know how to add helper text to an option?

You can use a custom option slot and check for the option's __CREATE__ prop:
https://github.com/vueform/multiselect#select-with-custom-options-slot


Testing semantic release

:tada: This PR is included in version 0.8.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


engine return bucket and fuel burn time multiplier

Resolved #64
Added fuel burn time multiplier config

I'll leave it to ewoudje to review this but I'd like to thank you for contributing to Eureka, I really appreciate it. I saw your previous PRs too and they're phenomenal, great work. Let me know if you need any help, you can also DM me on Discord.

maybe something about this didnt work; the engine just ate my bucket.. (eureka-1201-1.3.0-beta.1)

maybe something about this didnt work; the engine just ate my bucket.. (eureka-1201-1.3.0-beta.1)

eureka-1201-1.3.0-beta.1 does not contain this change.


[Suggestion] Add "Back to top" button in the schedule view.

When I browse the talks for day 1, at the end I have to scroll the entire page to the top to see the schedule for day 2. With a "back to top button" should be easier to do. Another solution is to add a button to switch directly to day 2.

True! I shall add one.

Closing, as we're restarting the project


Error on creation of PVC

2019-12-04T18:39:28.264Z	INFO	storageos.node	Failed to sync node labels	{"Request.Namespace": "", "Request.Name": "chichi-master-server", "error": "API error (Server failed to process your request. Was the data correct?): node not found"

Hi @includeamin this error is a result of the StorageOS operator attempting to sync k8s node labels for a node that is not running StorageOS and as such is not part of the StorageOS cluster. The error is nothing to worry about and does not relate to the creation of your PVC.
We are aware of this issue and are working on a fix.

it was working properly but after an hour not work anymore.
I had MongoDB deployment with that PVC.

Could you send an email to us with the troubleshooting information requested on the page below? If you could upload the cluster diagnostic that'd also be very helpful.
https://docs.storageos.com/docs/support/contactus

sorry, I don't have that cluster anymore.  after that, I set up a fresh cluster.


Secrets sent to server in plain text

The websites states the following:

If you include a passphrase (available under "Privacy Options"), we use it to encrypt the secret. We don't store the passphrase (only a bcrypted hash) so we can never know what the secret is because we can't decrypt it.

I created a secret with a passphrase and it sent both the secret and passphrase in plain text to the server in a HTTP POST request.

I can confirm this. This is painful.

We don't store the passphrase

The websites states it isn't stored, that doesn't specify anything about receiving the secret.

so we can never know what the secret is because we can't decrypt it.

but then I think this is a bit of false advertising.
I agree this encryption/decryption step must be done exclusively on the client for it to make any sense.

I'm genuinely curious, care to elaborate? @timvisee
Not specific to this project, are you arguing that, for example, sending a login password in plain text to a server over HTTPS is a bad idea? Or are you saying that we can't know for sure what OTS does with your plain-text secret, beside their claims?

I followed the links btw and did not find a satisfactory answer.

A password needs to be processed on the server for authentication purposes. You're not increasing the security by hashing the password it on the client (then the hash turns into a password). However, for encryption of secrets that the server don't even need to know the plain text version for processing, you can improve the security by encrypting it as early as possible (on the client).

Or are you saying that we can't know for sure what OTS does with your plain-text secret, beside their claims?

Indeed. But why trust them if we can not give them access to the plain text data in the first place?

Really?