Flexible mapping to CLR types and members (Custom O/C Mapping)

Although we have had support for POCOs through a few major versions, using EF for persistence still places some undesired constraints on the design choices developers can make on their domain objects.
E.g, EF Core still requires that mapped CLR types can be instantiated through constructors without parameters, that they contain both property getters and setters for each mapped scalar and reference property, and that all mapped collection navigation properties are exposed as properties that are of a type that implements ICollection.
EF Core also requires each entity type in the model to be mapped to a distinct CLR type, which makes some dynamic model scenarios (#2282) harder.
Moreover, scalar properties on objects have to be of a small set of recognized types in order to be mapped (in EF Core the set of types supported natively by the provider).
Richer constructs such as collections of scalars or collections of complex types, ordered collections, inheritance in complex types, collection manipulation methods, factory methods, and immutable objects are not supported.
This issue tracks the removal of those constrains as a whole and serves as a parent issue for some individual features that we will track independently:

[x] #246 Complex types or value objects support
[ ] #752 Custom collection patterns
[ ] #2919 Richer collection support
[ ] #2968 Custom property access patterns
[ ] #242 Simple data store to CLR conversions (custom type mapping)
[ ] #3342 Flexible construction through factory methods and DI
[ ] #4179 Collections of scalar types

@rowanmiller Looking at this it seems that the bug I filed yesterday is more specific about property access while this is the all-encompassing flexible mapping. Would you mind if I just add #2968 as a child, similar to how #857 is a child specific to collection patterns?

@divega yep that works

Otherwise I can merge it all here...

Made some edits to the original text of the issue to add background and additional links to related issues.

Hi,
the following example is for Illustration how maybe interface definition could be used to map EF structures.
public class BloggingContext : DbContext
{
    // Type mapping for materialization
    public DbSet<IBlog, Blog> Blogs { get; set; }

    // Materialization can create type on the fly because no specific type was given. (Generators or emitting)
    public DbSet<IPost> Posts { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<IBlog>()
            .Property(b => b.Url);

        // Optional: provide type creation factory.
        modelBuilder.Entity<IBlog>(() => new Blog());

        // Optional: provide collection behavior if non is provided then use default or emitt or generate.
        modelBuilder.Collection<IList<Any>>((IList<Any> collection, Any item) => collection.Add(item));
    }
}

// Any represents all kind of types which can be used in a collection.
class Any { }

public class DbSet<T, TEntity> where TEntity : class, T { }

public interface IBlog
{
    int BlogId { get; set; }

    string Url { get; set; }

    IList<IPost> Posts { get; set; }
}

public class Blog : IBlog
{
    public int BlogId { get; set; }

    public string Url { get; set; }

    public IList<IPost> Posts { get; set; }
}

public interface IPost
{
    int PostId { get; set; }

    string Title { get; set; }

    string Content { get; set; }

    int BlogId { get; set; }

    IBlog Blog { get; set; }
}

public class Post : IPost
{
    public int PostId { get; set; }

    public string Title { get; set; }

    public string Content { get; set; }

    public int BlogId { get; set; }

    public IBlog Blog { get; set; }
}

I know there are lot of other scenarios, so if someone has specific questions how for example x,y and z would fit in this kind of mapping, please ask, I will try to provide examples and explanations.

Actually, todays code base is quite close to being able to handle this. It just needs to be 'laxed in some parts.
I got stuck at the type check in EntityType.AddProperty, I dunno if it would have worked otherwise (probably not...)
We need to be able to specify, when adding a Property:

Clr type (not needed in practice as PropertyInfo.SetValue takes object)
Resulting primitive backing type (for DB provider)
Getter (converts from Clr type to Resulting primitive type)
Setter (converts from Resulting primitive type to Clr type)

Here's a (hopefully somewhat) functioning example of what I'd like to achieve:
public class Context : DbContext
{
    private static readonly IServiceProvider _serviceProvider = new ServiceCollection()
        .AddEntityFrameworkSqlServer()
        .AddSingleton<ICoreConventionSetBuilder, MyCoreConventionSetBuilder>()
        .BuildServiceProvider();

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder
            .UseInternalServiceProvider(_serviceProvider)
            .UseSqlServer(@"Data Source=.\SQLEXPRESS2014;Initial Catalog=EfCoreTest;Integrated Security=True;");
    }

    private class MyCoreConventionSetBuilder : CoreConventionSetBuilder
    {
        public override ConventionSet CreateConventionSet()
        {
            var value = base.CreateConventionSet();
            value.EntityTypeAddedConventions.Add(new MyPropertyDiscoveryConvention());
            return value;
        }
    }

    private class MyPropertyDiscoveryConvention : IEntityTypeConvention
    {
        public InternalEntityTypeBuilder Apply(InternalEntityTypeBuilder entityTypeBuilder)
        {
            entityTypeBuilder.Metadata.AddProperty("Ancestors", propertyType: typeof(string), flexible: true, serialize: JsonConvert.Serialize, parse: value => JsonConvert.Parse<List<int>>(value));

            return entityTypeBuilder;
        }
    }

    public DbSet<Page> Pages { get; set; }

    public class Page
    {
        public int Id { get; set; }
        public List<int> Ancestors { get; set; }
    }
}

@rowanmiller any thoughts or progress on this? I really think that just relaxing the checks a bit on this (and maybe generalizing the structure of Property a little) would get us 80% there.
( @divega )
Also, FWIW, I now think that we would be better off with a new AddFlexibleProperty method rather than further overloading the AddProperty.

@rowanmiller what is blocking to implement this. Maybe, I can help with this?

My example of where this would be useful would be something like the following:
public class BlogPost
{
    // Other properties omitted for brevity...

    public TagSet Tags { get; set; }
}

In my database, I want to store tags as a flat list (not a relational table), purely for lookup efficiency. It's simple to tests like WHERE [Tags] LIKE '%|MyTag%' to return values, etc.
With this in mind, I *could * form a type, like the following:
public struct TagSet : ICollection<string>
{
    private string _tagString;
    private Lazy<HashSet<string>> _collectionThunk;
    private bool _modified;
    private bool _hasValue;

    public TagSet(string tagString)
    {
        _tagString = tagString;
        _collectionThunk = new Lazy<HashSet<string>>(() => ResolveCollection(tagString));
        _modified = false;
        _hasValue = true;
    }

    public int Count => _collectionThunk.Value.Count;

    public bool IsReadOnly => false;

    public void Add(string item)
    {
        _collectionThunk.Value.Add(item);
        _modified = true;
    }

    public void Clear()
    {
        _tagString = null;
        _collectionThunk = new Lazy<HashSet<string>>(() => ResolveCollection(null));
        _modified = true;
    }

    public bool Contains(string item)
    {
        if (string.IsNullOrEmpty(_tagString))
        {
            return false;
        }

        if (_collectionThunk.IsValueCreated)
        {
            return _collectionThunk.Value.Contains(item);
        }

        var culture = CultureInfo.CurrentCulture;
        return culture.CompareInfo.IndexOf(_tagString, $"|{item}|", CompareOptions.IgnoreCase) >= 0;
    }

    public void CopyTo(string[] array, int arrayIndex) => _collectionThunk.Value.CopyTo(array, arrayIndex);

    public IEnumerator<string> GetEnumerator() => _collectionThunk.Value.GetEnumerator();

    public bool Remove(string item)
    {
        if (!string.IsNullOrEmpty(_tagString))
        {
            _modified = true;
            return _collectionThunk.Value.Remove(item);
        }
        return false;
    }

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    private static HashSet<string> ResolveCollection(string value)
    {
        var hashSet = new HashSet<string>(StringComparer.OrdinalIgnoreCase);

        if (!string.IsNullOrEmpty(value))
        {
            value = value.Trim('|');
            foreach (string tag in value.Split(new[] { '|' }, StringSplitOptions.RemoveEmptyEntries))
            {
                hashSet.Add(tag);
            }
        }

        return hashSet;
    }

    public static implicit operator string(TagSet tags)
    {
        if (tags._modified)
        {
            if (tags._collectionThunk.IsValueCreated)
            {
                if (tags._collectionThunk.Value.Count == 0)
                {
                    return null;
                }

                return $"|{string.Join("||", tags._collectionThunk.Value)}|";
            }
        }

        return tags._tagString;
    }

    public static implicit operator TagSet(string tagString) => new TagSet(tagString);

    public override string ToString() => this;

    public bool HasValue => _hasValue;
}

Which accepts a string, such like |Hello||World||These||Are My||Tags|, and allows lazy evaluation of those as an ICollection<string. It can react to changes and allow me to rebuild my flat tag string when I need it to.
I wondered how I would like this to integrate into EF, and perhaps a custom interface (taking some inspiration from AutoMapper):
public interface IValueMapper<TSource, TTarget>
{
    TTarget Map(TSource source);

    TSource Map(TTarget target);
}

public class TagSetValueMapper : IValueMapper<string, TagSet>
{
    public string Map(TagSet target) => target;

    public TagSet Map(string source) => new TagSet(source);
}

Perhaps we could then configure this as part of the EF model:
builder.Property(m => m.Tags).IsOptional().Maps<TagSet, string>(new TagSetValueMapper());

Honestly no ideas about how you would handle LINQ expressions though.
Thoughts? Would this be achievable? Perhaps even using a custom type converter instead? Who would be responsbile for handling DbNull checking though? EF or a mapper?

@Antaris not disagreeing that this is a valid scenario for this feature but could you elaborate more on the motivation? You said it is "purely for lookup efficiency" but my understanding is that this hinders searchability to make persistence and retrieval potentially more efficient,  e.g. with a regular index it is not possible to resolve WHERE [Tags] LIKE '%|MyTag|%', but once you apply this pattern it is no longer necessary to update andbquery an additional table (or tables) for the tags.
Re LINQ as you mentioned, it becomes more complicated. Two possible general approaches:

When you do something like this you are effectively opting out search efficiency so client evaluation is good enough. It will be up to you to make sure that your queries contain predicates on more than just Tags so you will never load the whole table into memory.
the mapping transformation (or a version of it) could be expressed in LINQ expressions rather than regular code so that references to Tags can be expanded into the expression before we attempt to compute the SQL translation.

@divega
Hmmm, I hadn't thought of that :-/ very good point on the indexes front. From previous experience I've found the flat list vs many-2-many tables setup to be more efficient in terms of manageability, hadn't really considered T-SQL index performance. I'll have to think about it and see about potential impacts. I guess I'd need to weigh up searchability vs storage efficiency :-/

@divega Out of interest I did some proof of concept work by fiddling with some of the internal API, and I got it working for the materialization point of view.
https://gist.github.com/Antaris/ca0f271311fdb84ef3a8db95b149a307
But, because my struct isn't immutable, the state manager doesn't pick up the change.

Updated my Gist, got expressions working but was definately tricky.
My TagSet implements ICollection<TagSet> and the ICollection<T>.Contains method is special cased in re-linq as a result operator (unlike string.Contains), so had to rename my method Exists and created a custom method translator for it.
Seems to work nicely.

Since this doesn't seem to be on the roadmap, I'd like to stop waiting and ask is this functionality available in any other framework?
EF6? (rather not but ...)
Maybe Hibernate.NET?

Guys this should be prioritized.
The whole point of an ORM is to Map to Objects... and EF has traditionally stood out particularly due to its ability to map to POCOs / domain representations of objects.
This means removing the friction of relational mapping and the ability to hide the complexity. Today there is still quite a LOT of friction in this and the reality is quite a ways from the promise/vision.
I know the team's been really focused on re-creating EF Core for modern computing paradigms (cloud, lightweight client, mobile – still missing, though really who cares – Realm has you beat)... but I feel it's really time to get back to new feature growth as opposed to catch-up/parity with EF6.
EF's biggest benefits are that of productivity by removing the object-relational friction / impedance mismatch and custom mapping would allow the framework to much better deliver on its promise.
Would love to see the team bite off this big, ambitious chunk and put the framework on an exciting path against the alternatives!
#keepupthegreatwork

hope flexible mapping also includes the possibility to totally bypass (turn off, remove) the property auto mapping feature. is this possible at the current stage?
if i'm having an o-o class and i'm extending it with properties i don't wanna write to DB, i need to go tot the mapping class and list there a property i don't want to map just to get it ignored.

@hidegh Agreed that would be super helpful. With the amount of limitations to EF object mapping we have a slew of properties outside of what's persisted. Would be much better to opt in to persisting them than opt out for everything else.
On that note, computed/get-only properties should not need to ever be explicitly ignored - by definition they can never persist lol. This should be automatically ignored by EF but right now you have to specify to ignore those too.

@marchy If you are finding read-only properties (i.e. properties without a setter) being mapped automatically, then can you please file a new issue with a repro. That should not be happening.

@bjorn-ali-goransson
i still wote for nhibernate. it's more mature and had features from 2007 that EF6 still has not. not to mention that it's architecture must be cleaner, cause a 3rd party fluent mapping api to NH is more powerful that the original one in EF6.
Actually I did a test (last week) and here's the result:
EF6 - due to convention I can do a protected internal virtual property mapping (with underscore prefix) and can automate to generate clean column names - but all the way how this is solved with conventions is too primitive (not as intuitive as with fluent NH). even collection encapsulation can be solved. so almost a hit, almost, because while using IQueryable, for includes you need to use not the main property but the "backing" one - and this is not an intuitive way to work with IQueryable.
EF7 - now it seems they don't map fields by default (but need to re-verify). anyhow it seemed perfect. I got the feeling of it, started to like it despite not having real conventions, missing NxN, group by not on the database level...   But when using NH/EF6 and not having DB connection and a lazy-load proxy is hit, guess what happens: right, you got an exception (so actually the missing lazy load meas missing exceptions). Now EF7 does not use proxies (nor custom nor 3rd parties). So anything not eager loaded is NULL or an empty collection.  If you think this is not a big issue, ask people who had similar issue with Linq2sql and wasted productive hours to find a bug...
Despite that in 2012 the main team left NHibernate, now it's managed still, but there are less frequent releases, it's still in a beter shape than any of the existings EF's backed by Microsoft. And just to be mean a bit: a framework, not production ready (perf. issues due group by, the mentioned lazy load issue), how can have a version number above or equal 1.0?
If someone asks me, I always tell this: to NH you got a nice manual how to do things...to EF you get documents on how to hack it, so that you get a near to ORM feeling of it. Sorry guys, EF7 might once be a nice product, but in v1.1 it is still not, despite having an open sourced ORM there for more than a decade you could learn from...

@hidegh NHibernate not having any async support is a dealbreaker for me.

@jnm2 a clean code will keep cost of quality low (less bugs, easier to maintain, easier to extend). considering prices for a developer vs. price for better hardware... btw. if you need performance, there's nothing wrong to have NHibernate for the command and views and EF (even database-first) with async for the read model.

I'm most familiar with writing desktop clients, and from personal experience, you need async saves. Otherwise you end up with UI lag or thread safety issues to work around. Still a dealbreaker for me.

NHibernate seems quite "dead", no?
I'll check out raven db while waiting for this.
lör 18 feb. 2017 kl. 15:09 skrev Joseph Musser<EMAIL_ADDRESS>
I'm most familiar with writing desktop clients, and from personal
experience, you need async saves. Otherwise you end up with UI lag or
thread safety issues to work around. Still a dealbreaker for me.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aspnet/EntityFramework/issues/240#issuecomment-280847821,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAoyAAxfl0LzwYZKTiKKkRViRdkzBakVks5rdvuIgaJpZM4B9gcA
.

Any progress on using readonly fields and properties with Entity Framework.  Currently I just want to pass in a Guid into the constructor which is stored in a readonly property ala #10400.
Maybe we could have a Materializer which could feed in mappings into some OnCreate where we could decide which constructor to call to materialize our entity?
class InvoiceEntityTypeConfiguration : IEntityTypeConfiguration<Invoice>
{
    public void Configure(EntityTypeBuilder<Invoice> builder)
    {
        builder.ToTable("Invoices", "dbo");
        builder.Materializer.OnCreate((PropertyMap pm) =>
        {
            return new Invoice(pm.Id, pm.Number);
        });
    }
}

Probably a dumb thought, but these kind of design decisions is killing me.  I don't want to duplicate an entity into its own state type if I can help it.

Which one of the issues in this issue's description handles support for unsigned integer types?

@smarts Unsigned integer types are supported as of EF Core 2.1.

@ajcvickers is there documentation on how to use this feature? My team recently tried to use them and was getting an exception that EF Core was expecting Int32 instead of UInt32. Do you know if this feature works for shadow properties too? Also, does it work for any DB integer type?
Our specific use-cases are SQL Server and PostgreSQL

@smarts Some databases natively support storing unsigned types, in which case they should behave like any other property.  In this case the value is stored in the database as a bigger signed type, by default.
If you have an entity type with unsigned properties and you are seeing errors, then please file a new issue with a small, runnable project/solution of complete code listing that demonstrates the behavior you are seeing.

@ajcvickers sorry for delaying my response.

In this case the value is stored in the database as a bigger signed type, by default.

Does that mean that using UInt32 for an entity's property will only work if the DB type for the corresponding column supports the full range of values for UInt32? I.e., we want to use INT in the DB and UInt32 on the entity.
While I understand that this conversion is generally bad, in the specific case of [generated] IDs the risk becomes negligible. Admittedly, this is based on my minimal experience with SQL DBs, so I apologize if this assumption is incorrect.
Is it possible to allow this generally-risky conversion for the negligible-risk scenario of IDs via some custom EF Convention?

@smarts take a look at value converters - you can set up lossy value conversions, but EF will not that automatically for you (as it does with uint->long). However, lossy value conversions may get a bit tricky in some scenarios, and are best avoided if you have an easy lossless alternative (such as long).

@roji As far as I can see, conversion between UInt32 and Int32 is lossless if you use "C-Style" casts. Negative int32 values are mapped to "high" unsigned values and back, one just should be careful when interpreting the converted values (e. G. ordering is different).

@markusschaber you're technically right: if your value converter does a simple cast in C#, then uint values which are larger than int.MaxValue will show up as negatives. While this is technically lossless and would work, as you say there are some tricky caveats.
Given that databases typically have a larger bigint type which can contain all uint values, it generally really makes sense to convert to that - trying to save 4 bytes isn't going to be justifiable in most cases.

@markusschaber i'm not sure if you noticed, but i'm specifically talking about auto-generated IDs in the database. Using long for the entity's property type has the same problem (allowing negative values), which isn't what I want. Also, as far as large values… since the value only ever comes from the DB (i.e., not set by any C# code outside of EF Core reading DB values), it will never be larger than Int32.MaxValue (because INT is the DB type).

Just to close the loop on this. When deciding how to convert automatically, we look at:

Is is lossy?
Does it preserve semantics on the database--most notably ordering?

The second point is why we convert  uint to long, and not uint to int, since for the latter, if the uint does have a value that can't "fit" it will still be stored, but it will be interpreted as negative by the database and hence will have a different ordering.
That being said, it's safe to convert uint to int if you know that the domain space for your uints can fit in an int, or if a specific ordering on the database is not needed. One way to do it is:
modelBuilder.Entity<Foo>().Property(e => e.MyUInt).HasConversion<int>();

Finally, there are currently some limitations for conversion on keys: see #11597

@ajcvickers - What exactly needs-cleanup label is?

Proposed for 6.0

Does it mean not earlier than November 2021?

@voroninp yes, 6.0 would mean November 2021. Previews would make this available much earlier, though.

I'm just here to say I've been waiting 6 years and counting for polymorphic relations using discriminators. I think it's more than made a case for itself from the digging I've been doing here.

Something along the lines of:
Integration
IIntegratable
User : IIIntegratable
Organization : IIntegratable
Therefore, the table integration in the database gets a integratable_type and integratable_id column, then I could go:
integration.Integratable...; // (returns a IIntegratable that can be passed through pattern matching to detect concrete type)
// and of course:
user.Integrations...; // (returns a list of `Integration` instances)


test/threadstest.c: Add checks for CRYPTO_THREAD_lock_new()

Add checks for the return value of CRYPTO_THREAD_lock_new() in order to avoid Null pointer dereference.
Fixes: 5f8b812931 ("Add locking to atomic operations in rw/rcu tests")
Fixes: d0e1a0ae70 ("RCU lock implementation")
Fixes: 71a04cfca0 ("Implement new multi-threading API")

Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

This pull request is ready to merge

Merged to the master and 3.3 branches. Thank you for your contribution.


Mistake in the official document

js sample code syntax is incorrect.
https://developers.google.com/actions/assistant/updates/daily#finalize_the_subscription
https://developers.google.com/actions/assistant/updates/notifications#finalize_the_subscription
// incorrect
appMap.set('finish.update.setup', function(app)) {
  if (app.isUpdateRegistered()) {
    app.tell("Ok, I'll start giving you daily updates.");
  } else {
    app.tell("Ok, I won't give you daily updates.");
  }
}

// correct
appMap.set('finish.update.setup', function(app) {
  if (app.isUpdateRegistered()) {
    app.tell("Ok, I'll start giving you daily updates.");
  } else {
    app.tell("Ok, I won't give you daily updates.");
  }
})

also , Is correct both appMap and actionMap? I think actionMap is better.
Please check this 🙏 thanks.

+1

This issue has been resolved with version 2 of the Node.js client library.


Python module not found

I have just started to familiarize myself with LEAN and some of the python examples on Github are using these import statements or similar ones and are not working:
from Portfolio.EqualWeightingPortfolioConstructionModel import EqualWeightingPortfolioConstructionModel
from Alphas.ConstantAlphaModel import ConstantAlphaModel
from Execution.ImmediateExecutionModel import ImmediateExecutionModel
from Risk.MaximumSectorExposureRiskManagementModel import MaximumSectorExposureRiskManagementModel

Perhaps these modules are now all imported via  QuantConnect.Algorithm.Framework.<moduleName>

Hi @pieroliviermarquis, what do you mean by "not working"?
Sometimes we need to rebuild Lean in order to the python framework modules are moved over to Lean/bin/Debug and be compiled when the engine imports the algorithm.
Note that we have C# and python version of the framework models/modules. The C# ones are imported via QuantConnect.Algorithm.Framework.<moduleTypeName> while the python ones are imported via <moduleTypeName>.<moduleName>. You can use either C# and python modules in python algorithms.


PD: TC Add presentational components for TC profiles based off dummy data

Overview
In an attempt to separate add/remove/nesting logic and presentational concerns. This ticket is for making the following components in charge of rendering TC profile steps and cycles:

[ ] ProfileStepsGroup (parent)
[ ] ProfileStepRow (to be used for delete step)
[ ] ProfileStepFields (presentational without field connector logic yet)
[ ] ProfileCycleRow (to be used for delete cycle)

Implementation details
Render out components above with the following mock redux data
const dummyData = {
    profileOrderedItemIds = ['a', 'x'], // a step, then a cycle containing 2 steps
    profileItemsById = {
        'a': {
            type: 'profileStep',
            id: 'a',
            name: 'Step A',
            temperature: '95',
            durationMinutes: '3',
            durationSeconds: '',
        },
        'b': {
            type: 'profileStep',
            id: 'b',
            name: 'Step B',
            temperature: '95',
            durationMinutes: '1',
            durationSeconds: '',
        },
        'c': {
            type: 'profileStep',
            id: 'c',
            name: 'Step C',
            temperature: '72',
            durationMinutes: '',
            durationSeconds: '30',
        },
        'x': {
            type: 'profileCycle',
            id: 'x',
            steps: ['b', 'c'],
            repetitions: '3',
        },
    }
}

Closing as it is no longer relevant


faster-whisper vs whisper: PyAV stops during decode, ffmpeg continues

The audio file is corrupted at the end, so an error is expected during decode process. However, PyAV stop processing while whisper using ffmpeg process the file until the corrupted are is detected.
Expected behavior: PyAV to process valid part of the file and throw an waning message.
Can we add a parameter to faster-whisper to behavior like whisper? i.e. process the file until the corrupt part.
Workaround: use ffmpeg command line to export corrupted file - discarding  invalid data - and run faster-whisper on top of new file processed by ffmpeg. This is a waste of processing if we can have a parameter to ignore corrupted content.

can you upload the file to test?

12487430.mp4a.gz
This is the python source code
`import time
from faster_whisper import WhisperModel
import logging
logging.basicConfig()
logging.getLogger("faster_whisper").setLevel(logging.DEBUG)
model_size = "large-v3"
model = WhisperModel(model_size, device="cuda", compute_type="float16")
def speech2text(fileName):
segments, info = model.transcribe(fileName, beam_size=5, language="pt")
text = ""
for segment in segments:
text = text + "[{0:.2f}s -> {1:.2f}s] {2}\n".format(segment.start, segment.end, segment.text)
text = text + "----\n"
print(text)
if name =="main":
start_time = time.time()
speech2text("/mnt/media/audio/v1/2024-09-02/12487430.mp4a")
print("--- %s seconds ---" % (time.time() - start_time))
print("Done!")
`
This is the python3 output
Traceback (most recent call last):   File "/mnt/bin/fastwhisper.py", line 21, in <module>     speech2text("/mnt/media/audio/v1/2024-09-02/12487430.mp4a")   File "/mnt/bin/fastwhisper.py", line 12, in speech2text     segments, info = model.transcribe(fileName, beam_size=5, language="pt")   File "/home/rodrigo/.local/lib/python3.10/site-packages/faster_whisper/transcribe.py", line 319, in transcribe     audio = decode_audio(audio, sampling_rate=sampling_rate)   File "/home/rodrigo/.local/lib/python3.10/site-packages/faster_whisper/audio.py", line 52, in decode_audio     for frame in frames:   File "/home/rodrigo/.local/lib/python3.10/site-packages/faster_whisper/audio.py", line 103, in _resample_frames     for frame in itertools.chain(frames, [None]):   File "/home/rodrigo/.local/lib/python3.10/site-packages/faster_whisper/audio.py", line 90, in _group_frames     for frame in frames:   File "/home/rodrigo/.local/lib/python3.10/site-packages/faster_whisper/audio.py", line 80, in _ignore_invalid_frames     yield next(iterator)   File "av/container/input.pyx", line 207, in decode   File "av/container/input.pyx", line 166, in demux   File "av/container/core.pyx", line 286, in av.container.core.Container.err_check   File "av/error.pyx", line 326, in av.error.err_check av.error.OSError: [Errno 5] Input/output error: '/mnt/media/audio/v1/2024-09-02/12487430.mp4a'
This is the output of ffmpeg (it shows the Input/output error but export the file to /tmp/1.mp3
ffmpeg -v debug -i /mnt/media/audio/v1/2024-09-02/12487519.mp4a /tmp/1.mp3
 ffmpeg version 4.4.2-0ubuntu<IP_ADDRESS> Copyright (c) 2000-2021 the FFmpeg developers   built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)   configuration: --prefix=/usr --extra-version=0ubuntu<IP_ADDRESS> --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared   libavutil      56. 70.100 / 56. 70.100   libavcodec     58.134.100 / 58.134.100   libavformat    58. 76.100 / 58. 76.100   libavdevice    58. 13.100 / 58. 13.100   libavfilter     7.110.100 /  7.110.100   libswscale      5.  9.100 /  5.  9.100   libswresample   3.  9.100 /  3.  9.100   libpostproc    55.  9.100 / 55.  9.100 Splitting the commandline. Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'. Reading option '-i' ... matched as input url with argument '/mnt/media/audio/v1/2024-09-02/12487519.mp4a'. Reading option '/tmp/1.mp3' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option v (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input url /mnt/media/audio/v1/2024-09-02/12487519.mp4a. Successfully parsed a group of options. Opening an input file: /mnt/media/audio/v1/2024-09-02/12487519.mp4a. [NULL @ 0x561af38fb680] Opening '/mnt/media/audio/v1/2024-09-02/12487519.mp4a' for reading [file @ 0x561af38fc300] Setting default whitelist 'file,crypto,data' [aac @ 0x561af38fb680] Format aac probed with size=32768 and score=50 [aac @ 0x561af38fb680] Before avformat_find_stream_info() pos: 229 bytes read:65696 seeks:4 nb_streams:1 [aac @ 0x561af38fb680] All info found [aac @ 0x561af38fb680] Estimating duration from bitrate, this may be inaccurate [aac @ 0x561af38fb680] After avformat_find_stream_info() pos: 9852 bytes read:65696 seeks:4 frames:50 Input #0, aac, from '/mnt/media/audio/v1/2024-09-02/12487519.mp4a':   Duration: 00:03:02.12, bitrate: 66 kb/s   Stream #0:0, 50, 1/28224000: Audio: aac (LC), 44100 Hz, stereo, fltp, 66 kb/s Successfully opened the file. Parsing a group of options: output url /tmp/1.mp3. Successfully parsed a group of options. Opening an output file: /tmp/1.mp3. File '/tmp/1.mp3' already exists. Overwrite? [y/N] Y [file @ 0x561af392f500] Setting default whitelist 'file,crypto,data' Successfully opened the file. Stream mapping:   Stream #0:0 -> #0:0 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream) detected 12 logical cores [graph_0_in_0_0 @ 0x561af391f580] Setting 'time_base' to value '1/44100' [graph_0_in_0_0 @ 0x561af391f580] Setting 'sample_rate' to value '44100' [graph_0_in_0_0 @ 0x561af391f580] Setting 'sample_fmt' to value 'fltp' [graph_0_in_0_0 @ 0x561af391f580] Setting 'channel_layout' to value '0x3' [graph_0_in_0_0 @ 0x561af391f580] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x3 [format_out_0_0 @ 0x561af3a002c0] Setting 'sample_fmts' to value 's32p|fltp|s16p' [format_out_0_0 @ 0x561af3a002c0] Setting 'sample_rates' to value '44100|48000|32000|22050|24000|16000|11025|12000|8000' [format_out_0_0 @ 0x561af3a002c0] Setting 'channel_layouts' to value '0x4|0x3' [AVFilterGraph @ 0x561af392fb00] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed Output #0, mp3, to '/tmp/1.mp3':   Metadata:     TSSE            : Lavf58.76.100   Stream #0:0, 0, 1/44100: Audio: mp3, 44100 Hz, stereo, fltp, delay 1105     Metadata:       encoder         : Lavc58.134.100 libmp3lame cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)     Last message repeated 2 times /mnt/media/audio/v1/2024-09-02/12487519.mp4a: Input/output error2x     [out_0_0 @ 0x561af3a00a80] EOF on sink link out_0_0:default. No more output streams to write to, finishing. [libmp3lame @ 0x561af390dcc0] Trying to remove 815 more samples than there are in the queue size=    2821kB time=00:03:00.48 bitrate= 128.0kbits/s speed=96.9x     video:0kB audio:2821kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.008759% Input file #0 (/mnt/media/audio/v1/2024-09-02/12487519.mp4a):   Input stream #0:0 (audio): 7773 packets read (1509612 bytes); 7773 frames decoded (7959552 samples);    Total: 7773 packets (1509612 bytes) demuxed Output file #0 (/tmp/1.mp3):   Output stream #0:0 (audio): 6910 frames encoded (7959552 samples); 6911 packets muxed (2888515 bytes);    Total: 6911 packets (2888515 bytes) muxed 7773 frames successfully decoded, 0 decoding errors [AVIOContext @ 0x561af3932dc0] Statistics: 2 seeks, 13 writeouts [AVIOContext @ 0x561af3904680] Statistics: 1542770 bytes read, 4 seeks
This is the output for whisper
whisper --verbose True --language pt --model large /mnt/media/audio/v1/2024-09-02/12487519.mp4a
/home/rodrigo/.local/lib/python3.10/site-packages/whisper/__init__.py:146: FutureWarning: You are using torch.loadwithweights_only=False(the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value forweights_onlywill be flipped toTrue. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=Truefor any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.   checkpoint = torch.load(fp, map_location=device) [00:00.000 --> 00:28.000]  Música [00:30.000 --> 00:40.000]  Música [01:00.000 --> 01:10.000]  Música [01:10.000 --> 01:14.000]  Música [01:30.000 --> 01:40.000]  Música [01:40.000 --> 01:46.000]  Música [02:00.000 --> 02:21.040]  Super Mix, quarenta minutos de música na Mix. [02:30.000 --> 02:59.980]  Super Mix, quarenta minutos de música na Mix. [03:00.000 --> 03:00.460]  Música


Return to desktop on IL2 Great battle

My game comes back to the desktop after a few seconds, if I change a setting on OpenXR Toolkit.
But it works in safe mode.
[OXRTK] 2023-04-07 11:23:03 +0200: OpenXR Toolkit - GA-3 (v1.3.0)
[OXRTK] 2023-04-07 11:23:03 +0200: dllHome is "C:\Program Files\OpenXR-Toolkit"
[OXRTK] 2023-04-07 11:23:03 +0200: OpenXR-Toolkit layer is active
[OXRTK] 2023-04-07 11:23:03 +0200: Using layer: XR_APILAYER_NOVENDOR_XRNeckSafer
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_win32_convert_performance_counter_time
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_D3D11_enable
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_D3D12_enable
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_composition_layer_depth
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_composition_layer_color_scale_bias
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_unbounded_reference_space
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_spatial_anchor
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_perception_anchor_interop
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_win32_appcontainer_compatible
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_conformance_automation
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_composition_layer_reprojection
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_spatial_graph_bridge
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_spatial_anchor_persistence
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_palm_pose
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_scene_understanding
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_scene_understanding_serialization
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_FB_display_refresh_rate
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_eye_gaze_interaction
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_hand_interaction
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_hand_tracking
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_hand_joints_motion_range
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_hand_tracking_mesh
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_MSFT_controller_model
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_KHR_visibility_mask
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_samsung_odyssey_controller
[OXRTK] 2023-04-07 11:23:03 +0200: Runtime supports extension: XR_EXT_hp_mixed_reality_controller
[OXRTK] 2023-04-07 11:23:03 +0200: Requesting extra extension: XR_EXT_eye_gaze_interaction
[OXRTK] 2023-04-07 11:23:03 +0200: Requesting extra extension: XR_EXT_hand_tracking
[OXRTK] 2023-04-07 11:23:03 +0200: Requesting extra extension: XR_KHR_visibility_mask
[OXRTK] 2023-04-07 11:23:03 +0200: Requesting extra extension: XR_KHR_win32_convert_performance_counter_time
[OXRTK] 2023-04-07 11:23:03 +0200: Application name: 'OpenComposite_Il-2', Engine name: ''
[OXRTK] 2023-04-07 11:23:03 +0200: Detected OpenComposite
[OXRTK] 2023-04-07 11:23:03 +0200: Using OpenXR runtime Windows Mixed Reality Runtime 112.2211.2002
[OXRTK] 2023-04-07 11:23:03 +0200: SAFE MODE IS ENABLED! NO SETTINGS ARE LOADED!
[OXRTK] 2023-04-07 11:23:03 +0200: HAGS is on
[OXRTK] 2023-04-07 11:23:03 +0200: Using OpenXR system Windows Mixed Reality
[OXRTK] 2023-04-07 11:23:03 +0200: Ignoring XR_MSFT_hand_interaction for Windows Mixed Reality Runtime 112.2211.2002
[OXRTK] 2023-04-07 11:23:03 +0200: Using OpenXR resolution (no upscaling): 3156x3088
[OXRTK] 2023-04-07 11:23:04 +0200: Using Direct3D 11 on adapter: NVIDIA GeForce RTX 2080
[OXRTK] 2023-04-07 11:23:04 +0200: Early initializeInterceptor() call
[OXRTK] 2023-04-07 11:23:04 +0200: Open menu with CTRL+F2
[OXRTK] 2023-04-07 11:23:05 +0200: Projection calibration: 0.04909, -0.01023 | -0.04947, -0.00923
[OXRTK] 2023-04-07 11:23:14 +0200: D3D11Device destroyed
[OXRTK] 2023-04-07 11:23:14 +0200: Session destroyed
[OXRTK] 2023-04-07 11:23:14 +0200: Using Direct3D 11 on adapter: NVIDIA GeForce RTX 2080
[OXRTK] 2023-04-07 11:23:14 +0200: Early initializeInterceptor() call
[OXRTK] 2023-04-07 11:23:14 +0200: Open menu with CTRL+F2
[OXRTK] 2023-04-07 11:23:14 +0200: Projection calibration: 0.04909, -0.01023 | -0.04947, -0.00923
[OXRTK] 2023-04-07 11:23:14 +0200: Creating swapchain with dimensions=3160x3088, arraySize=1, mipCount=1, sampleCount=1, format=29, usage=0x31
[OXRTK] 2023-04-07 11:23:15 +0200: Creating swapchain with dimensions=3160x3088, arraySize=1, mipCount=1, sampleCount=1, format=29, usage=0x31
[OXRTK] 2023-04-07 11:24:06 +0200: Opening menu
[OXRTK] 2023-04-07 11:44:19 +0200: D3D11Device destroyed
[OXRTK] 2023-04-07 11:44:19 +0200: Session destroyed
Environment

Application/Game and version: IL2 great battle 5.101b lastest version
Make and model of VR device: Hp reverb G1

To Reproduce
Steps to reproduce the behavior:

Run '....' game
Enable option 'turbo mode on Cas 100% fixed foveated redering performance wide'
See error
Signature du problème
Nom d’événement du problème :	AppHangB1
Application Name:	Il-2.exe
Application Version:	<IP_ADDRESS>
Application Timestamp:	642c407f
Hang Signature:	b9e0
Hang Type:	134217728
Version du système:	10.0.19<IP_ADDRESS>.768.101
Identificateur de paramètres régionaux:	1036
Additional Hang Signature 1:	b9e005d1df50c717448e294572755889
Additional Hang Signature 2:	841f
Additional Hang Signature 3:	841f35523168f3ae7ad367234e057e5d
Additional Hang Signature 4:	b9e0
Additional Hang Signature 5:	b9e005d1df50c717448e294572755889
Additional Hang Signature 6:	841f
Additional Hang Signature 7:	841f35523168f3ae7ad367234e057e5d

or
Signature du problème
Nom d’événement du problème :	APPCRASH
Nom de l’application:	Il-2.exe
Version de l’application:	<IP_ADDRESS>
Horodatage de l’application:	642c407f
Nom du module défaillant:	nvwgf2umx.dll
Version du module défaillant:	<IP_ADDRESS>41
Horodateur du module défaillant:	641398c6
Code de l’exception:	c0000005
Décalage de l’exception:<PHONE_NUMBER>39b338
Version du système:	10.0.19<IP_ADDRESS>.768.101
Identificateur de paramètres régionaux:	1036
Information supplémentaire n° 1:	02ad
Information supplémentaire n° 2:	02add1cd6d082459141edd07550797a6
Information supplémentaire n° 3:	4e71
Information supplémentaire n° 4:	4e71970c65f05038c3dd672717229d2f
Expected behavior
A clear and concise description of what you expected to happen.
Game files verified
verified drivers
windows up to date
NVIDIA System Information 04-07-2023 16-03-10.txt
Screenshots
If applicable, add screenshots to help explain your problem. Use the Companion app to enable screenshots, then use Ctrl+F12 to capture. The files are placed in %LocalAppData%\OpenXR-Toolkit\screenshots or can be opened from the OpenXR Toolkit Companion app.

Enable option 'turbo mode on Cas 100% fixed foveated redering performance wide'

Fixed Foveated Rendering isn't supported with IL2. It shouldn't even be offered, but I assume you went and disabled the "Foveated Rendering killswitch" while in Safe mode to force it.
This is likely the cause of your problems. Please do a full settings reset to clear the killswitch flag back to its default and fully disable Foveated Rendering.

OK thanks
If I understand correctly, on the menu in safe mode, the kill switch must be ON?
Is this a function that we can use in DCS?

If I understand correctly, on the menu in safe mode, the kill switch must be ON?

Yes. This is what it defaults to for IL-2, and changing that was not a good idea.

Is this a function that we can use in DCS?

Yes FFR, works fine in DCS.


FIX: failed to remove policy from non-filled casbin model.

If someone using model like the RBAC model that not fill 6 slot(v0-v5), while adding the policy to the database(I'm testing with postgresql) the empty slot will be treat as NULL.
In adapter.py#L63 and adapter.py#L73 are constructing the query with ""(empty string) but None(NULL). It will never match any policy in this situation, so policies will never delete(But casbin will delete them from its in-memory model which will break the consistency.)
model e.g.
_model = """
[request_definition]
r = sub, obj, act

[policy_definition]
p = sub, obj, act, eft

[role_definition]
g = _, _

[policy_effect]
e = some(where (p.eft == allow)) && !some(where (p.eft == deny))

[matchers]
m = g(r.sub, p.sub) && keyMatch(r.obj, p.obj) && keyMatch(r.act, p.act)
"""

@thearchitector

@hsluoyz @DarcJC Thanks for bringing this to my attention, but the implementation you proposed only addresses the problem in one method, and because of the None type also fails the type check. Closing in favor of https://github.com/thearchitector/casbin-tortoise-adapter/commit/58444d9afcba4d23af461d5d4f104050f1b97733.


Teams exclusions - .txt files ??

Please provide the https://docs.microsoft.com/en-us/azure/architecture/example-scenario/wvd/windows-virtual-desktop-fslogix#teams-exclusions in the format of https://docs.microsoft.com/en-us/fslogix/manage-profile-content-cncpt#structure-of-redirectionsxml-file
Saying .txt files is very vague!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6386653c-9f82-1ea7-8f29-377be4e5e3b7
Version Independent ID: a5db2b47-afaf-5917-40cd-740168cad8b6
Content: FSLogix for the enterprise - Azure Architecture Guide
Content Source: docs/example-scenario/wvd/windows-virtual-desktop-fslogix.md
Service: architecture-center
Sub-service: azure-guide
GitHub Login: @christiaanbrinkhoff
Microsoft Alias: pnp

@DavidTaig
Thanks for your feedback! We will investigate and update as appropriate.

Approved. Please remove the .txt files under the Teams exclusions section.

Can you send us a PR on this @christiaanbrinkhoff ?

.txt files has been removed from https://docs.microsoft.com/en-us/azure/architecture/example-scenario/wvd/windows-virtual-desktop-fslogix#teams-exclusions Thanks :)

This looks to have been addressed in updates to the article.


test(conformance): Print difference of error codes to a file

Description:
This PR make the conformance test suite print the error codes of required errors and excess errors to a file.
Related issue:

Closes https://github.com/dudykr/stc/issues/709.

This is huge, it's going to a save time. is it possible to also log the locations for the errors?

Of course, I'll create another PR

@infix I made https://github.com/dudykr/stc/pull/711

Thank you!


Video capture only works after settings change

When I restart my TV, the video capture only starts working after I open PicCap, change a setting (e.g. the resolution) and save it. If I don't do that only UI gets captures (the WebOS menu or YouTube menu for example). Enabling/disabling QuickStart+ does not make any difference.
TV Model: 55SM8500
Software Version: 5.30.25

Having the exact same issue. It's, most of the time, fixed after restarting the service altogether in Piccap.
When disabling Quickstart+ I receive a toast notification that the system entered failsafe mode. Before the HBC update, disabling Quickstart+ worked and Piccap did what it had to do...
Any help would be greatly appreciated!

Same here, after TV starts, Piccap seems not to be autostarted. Just after manually opening it and start/restart the service, it works.


Under General Configuration- Missing configuration check and reloading options in HA 0.97

Home Assistant release with the issue:

Last working Home Assistant release (if known):
0,96,5
Operating environment (Hass.io/Docker/Windows/etc.):

Component/platform:

Description of problem:
After upgrading from HA 0.96.5 to HA 0.97 i see no more options to check configuration or to restart/reload anything. Reverting back to HA 0.96.5 I regain those options again.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):

Additional information:

Configuration > Server control
From the release notes.
https://www.home-assistant.io/blog/2019/08/07/release-97/#ui-changes

If its working, please close the issue


Update sbt to 1.5.5

Updates org.scala-sbt:sbt from 1.2.8 to 1.5.5.
Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

labels: library-update, semver-minor

Superseded by #106.


[docs] Add Chip TypeScript demo for Chip array

[x] I have followed (at least) the PR section of the contributing guide.

Add Chips Array Typescript demo for #14897

@sperry94 Much appreciated, thanks!


Realign Convert Buttons

Because the dividing space between the two editors got wider, the space between the two convert buttons no longer aligns. It really bugs me and was easy to fix, so I made a PR for it. I'll add screenshots of the differences, but you can also see the difference by looking at https://fshschool.org/FSHOnline/#/ and this branch.
On the deployed FSH Online site now, the space between the buttons does not align with the space between the editors:

On this branch, it does align:

I think they might still be a hair off, but this feels much better to me.

Plz ignore the massive blunder above

I guess I have no choice but to review this now.  Julia says I must.

Would it drive you crazy if I posted this super zoomed version that shows the actual "hair" of difference?

Yes.
But also, I did do that crazy zoomed in thing which is why I knew the hair difference was there, and I was hoping no one else would investigate the hair difference. I also tried briefly to fix it, but was unsuccessful. I'll try one more thing.

when I run npm install, I am getting a different package-lock.json than what is checked in.

Ugh I'm not sure if this has to do with me or not, but this is not the first time it's happened in this repo and I'm not sure why it keeps happening. I'll at least run npm install and check in the changes.

Thanks to @ngfreiter the difference is gone! 😅 It actually looks even better than it did in the pre-resizable editor world.
I also added the changes from npm install but I keep having issues with my npm install not getting all the changes, so if you still see differences, let me know.

I'm shocked that fractional pixels seems to be the one thing that is supported uniformly across all browsers.

Yes, you can specify fractional pixels. As this has been part of CSS since the very first version, it should be well supported by any browser that supports CSS at all.

However, I'll let the internet just do it's thing this time around.


Create Piumika9631.html

1st edit

Your pull request seems good, but must be checked by a human maintainer for the following reason(s):

This contains potentially hazardous tags that need manual review.

Please be patient until a maintainer has time to look over your contribution.
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

b896fe024cd50489e1d5508de0afd31496c6444c

Solve PR mistakes in the same PR.


Zone 0 gets wiped

Firstly thanks for your work.
I have two displays working with two wemo d1s. Both running firmware 2. Same power supplies. One of them works perfectly. The other one stops working and when i login it has no zones set everytime. I go back set zone 0 and it works again for a random period of time and then does the same thing. It doesnt reboot it just loses zone setting. I have resoldered wires incase of that being the issue but it has not fixed the problem.
Any idea what this might be?  I did try the latest firmware but it didnt seem to fix the issue.

How many zones are in use?
What are the operating modes for each zone?
It looks like the device is rebooting due to lack of memory. Or the device is damaged. If possible, try replacing the ESP module on the problem device.

I swapped wemo for another and no problem so was faukty hardware. Thanks can close.


Test Improvements and up code coverage

Pull Request Test Coverage Report for Build 25cc52582692ccaf524d5ee3650b62865eec8d5b-PR-8

4 of 4 (100.0%)  changed or added relevant lines in 1 file are covered.
8 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.4%) to 78.425%

Files with Coverage Reduction
New Missed Lines
%

VBaseProject.Api/Middleware/VBaseProjectMiddleware.cs
8
27.78%

Totals

Change from base Build 38730b5b0b7a563ffeea3100d7bd802dc381f263:
0.4%

Covered Lines:
630

Relevant Lines:
784

💛  - Coveralls


Fix Code Generator emitting uncompilable code for exception named 'Exception'

by alias-importing the base type kotlin.Exception.
The root cause was that the builder tries to reference the outer type (the thrift Exception) but gets kotlin.Exception instead.
This introduces kotest-assertions-compiler for not just this, but all tests in KotlinCodeGeneratorTest.kt to check if the resulting code compiles.
Doing this still has the potential for a clash if you define a type KotlinException in thrift, but that is maybe more unlikely?
Looking forward to your feedback!

Hi, thanks for the PR! That's a rather unfortunate bug. I'm not exactly sure how this aliased import is used, does it just hide kotlin.Exception?

this is what the generated code looks like with the aliasing:
package com.test

import com.microsoft.thrifty.Struct
import com.microsoft.thrifty.TType
import com.microsoft.thrifty.ThriftField
import com.microsoft.thrifty.kotlin.Adapter
import com.microsoft.thrifty.protocol.Protocol
import com.microsoft.thrifty.util.ProtocolUtil
import kotlin.String
import kotlin.jvm.JvmField
import kotlin.Exception as KotlinException

data class Exception(
  @JvmField
  @ThriftField(fieldId = 1)
  val message_: String?
) : KotlinException(), Struct {
  override fun write(protocol: Protocol) {
    ADAPTER.write(protocol, this)
  }

  private class ExceptionAdapter : Adapter<Exception> {
    override fun read(protocol: Protocol): Exception {
      var _local_message: String? = null
      protocol.readStructBegin()
      while (true) {
        val fieldMeta = protocol.readFieldBegin()
        if (fieldMeta.typeId == TType.STOP) {
          break
        }
        when (fieldMeta.fieldId.toInt()) {
          1 -> {
            if (fieldMeta.typeId == TType.STRING) {
              val message_ = protocol.readString()
              _local_message = message_
            } else {
              ProtocolUtil.skip(protocol, fieldMeta.typeId)
            }
          }
          else -> ProtocolUtil.skip(protocol, fieldMeta.typeId)
        }
        protocol.readFieldEnd()
      }
      protocol.readStructEnd()
      return Exception(message_ = _local_message)
    }

    override fun write(protocol: Protocol, struct: Exception) {
      protocol.writeStructBegin("Exception")
      if (struct.message_ != null) {
        protocol.writeFieldBegin("message", 1, TType.STRING)
        protocol.writeString(struct.message_)
        protocol.writeFieldEnd()
      }
      protocol.writeFieldStop()
      protocol.writeStructEnd()
    }
  }

  companion object {
    @JvmField
    val ADAPTER: Adapter<Exception> = ExceptionAdapter()
  }
}

It seems like a safer fix would be for generated code to use the fully-qualified name of the outer type. We might be able to do that by explicitly importing kotlin.Exception - as I understand KotlinPoet, that should be enough to surface the name clash and to force the use of the fully-qualified name.

Yes, that sounds safer, however I don't understand exactly what you mean by 'explicitly importing'. Can you provide some more guidance?

Regardless of how we ultimately address the bug, I'll be grateful for getting the tests passing again. Thanks again!

Sure, I was a bit overeager at first and added the alias to all FileSpecs, thinking it would only apply when the type that is being aliases is used. That is not the case so tests that had assertions on the complete generated code would stumble over the new import (that was totally useless). I fixed that.
Maybe related (but maybe not): https://github.com/square/kotlinpoet/issues/938

Hi, thanks for the PR! That's a rather unfortunate bug. I'm not exactly sure how this aliased import is used, does it just hide kotlin.Exception?

this is what the generated code looks like with the aliasing:
package com.test

import com.microsoft.thrifty.Struct
import com.microsoft.thrifty.TType
import com.microsoft.thrifty.ThriftField
import com.microsoft.thrifty.kotlin.Adapter
import com.microsoft.thrifty.protocol.Protocol
import com.microsoft.thrifty.util.ProtocolUtil
import kotlin.String
import kotlin.jvm.JvmField
import kotlin.Exception as KotlinException

data class Exception(
  @JvmField
  @ThriftField(fieldId = 1)
  val message_: String?
) : KotlinException(), Struct {
  override fun write(protocol: Protocol) {
    ADAPTER.write(protocol, this)
  }

  private class ExceptionAdapter : Adapter<Exception> {
    override fun read(protocol: Protocol): Exception {
      var _local_message: String? = null
      protocol.readStructBegin()
      while (true) {
        val fieldMeta = protocol.readFieldBegin()
        if (fieldMeta.typeId == TType.STOP) {
          break
        }
        when (fieldMeta.fieldId.toInt()) {
          1 -> {
            if (fieldMeta.typeId == TType.STRING) {
              val message_ = protocol.readString()
              _local_message = message_
            } else {
              ProtocolUtil.skip(protocol, fieldMeta.typeId)
            }
          }
          else -> ProtocolUtil.skip(protocol, fieldMeta.typeId)
        }
        protocol.readFieldEnd()
      }
      protocol.readStructEnd()
      return Exception(message_ = _local_message)
    }

    override fun write(protocol: Protocol, struct: Exception) {
      protocol.writeStructBegin("Exception")
      if (struct.message_ != null) {
        protocol.writeFieldBegin("message", 1, TType.STRING)
        protocol.writeString(struct.message_)
        protocol.writeFieldEnd()
      }
      protocol.writeFieldStop()
      protocol.writeStructEnd()
    }
  }

  companion object {
    @JvmField
    val ADAPTER: Adapter<Exception> = ExceptionAdapter()
  }
}

It seems like a safer fix would be for generated code to use the fully-qualified name of the outer type. We might be able to do that by explicitly importing kotlin.Exception - as I understand KotlinPoet, that should be enough to surface the name clash and to force the use of the fully-qualified name.

Yes, that sounds safer, however I don't understand exactly what you mean by 'explicitly importing'. Can you provide some more guidance?

Regardless of how we ultimately address the bug, I'll be grateful for getting the tests passing again. Thanks again!

Sure, I was a bit overeager at first and added the alias to all FileSpecs, thinking it would only apply when the type that is being aliases is used. That is not the case so tests that had assertions on the complete generated code would stumble over the new import (that was totally useless). I fixed that.
Maybe related (but maybe not): https://github.com/square/kotlinpoet/issues/938

Maybe related (but maybe not): square/kotlinpoet#938

Good find!  This does indeed seem related.  The recent comment by Zac seems to indicate that the solution I was proposing may not actually work.  Let me do a bit of exploration - I still feel like there's a safer option than import aliases here.

Maybe related (but maybe not): square/kotlinpoet#938

Good find!  This does indeed seem related.  The recent comment by Zac seems to indicate that the solution I was proposing may not actually work.  Let me do a bit of exploration - I still feel like there's a safer option than import aliases here.

Definitely an upstream issue, I found out the following funny repro case:
import com.squareup.kotlinpoet.ClassName
import com.squareup.kotlinpoet.FileSpec
import com.squareup.kotlinpoet.FunSpec
import com.squareup.kotlinpoet.TypeSpec

val namespace = "test"

val kotlinExceptionName = ClassName("kotlin", "Exception")
val customExceptionName = ClassName(namespace, "Exception")
val customException = TypeSpec
        .classBuilder("Exception")
        .superclass(kotlinExceptionName)
        .addFunction(
                FunSpec
                        .builder("test")
                        .addParameter("e", customExceptionName)
                        .build()
        )
        .build()

FileSpec.builder(namespace, "a.kt")
        .addType(customException)
        .build()
        .toString()

val funn = FunSpec.builder("test")
        .addParameter("e", customExceptionName)
        .addParameter("f", kotlinExceptionName)
        .build()

FileSpec.builder(namespace, "b.kt")
        .addType(customException)
        .addFunction(funn)
        .build()
        .toString()

this outputs
package test

import kotlin.Exception

class Exception : Exception() {
  fun test(e: Exception) {
  }
}

and
package test

class Exception : kotlin.Exception() {
  fun test(e: Exception) {
  }
}

fun test(e: Exception, f: kotlin.Exception) {
}

So as soon as I use both colliding types in the same function, the anti-collision works as expected. 🤷
I will report this upstream.

Definitely an upstream issue, I found out the following funny repro case:
import com.squareup.kotlinpoet.ClassName
import com.squareup.kotlinpoet.FileSpec
import com.squareup.kotlinpoet.FunSpec
import com.squareup.kotlinpoet.TypeSpec

val namespace = "test"

val kotlinExceptionName = ClassName("kotlin", "Exception")
val customExceptionName = ClassName(namespace, "Exception")
val customException = TypeSpec
        .classBuilder("Exception")
        .superclass(kotlinExceptionName)
        .addFunction(
                FunSpec
                        .builder("test")
                        .addParameter("e", customExceptionName)
                        .build()
        )
        .build()

FileSpec.builder(namespace, "a.kt")
        .addType(customException)
        .build()
        .toString()

val funn = FunSpec.builder("test")
        .addParameter("e", customExceptionName)
        .addParameter("f", kotlinExceptionName)
        .build()

FileSpec.builder(namespace, "b.kt")
        .addType(customException)
        .addFunction(funn)
        .build()
        .toString()

this outputs
package test

import kotlin.Exception

class Exception : Exception() {
  fun test(e: Exception) {
  }
}

and
package test

class Exception : kotlin.Exception() {
  fun test(e: Exception) {
  }
}

fun test(e: Exception, f: kotlin.Exception) {
}

So as soon as I use both colliding types in the same function, the anti-collision works as expected. 🤷
I will report this upstream.

I validated that the upstream fix actually fixes this issue - unfortunately no release yet. This can be closed - in case you want to keep the shouldCompile assertions this also introduced, I can open another PR with only that.

I validated that the upstream fix actually fixes this issue - unfortunately no release yet. This can be closed - in case you want to keep the shouldCompile assertions this also introduced, I can open another PR with only that.

Sorry, life got hectic and I didn't check notifications for quite some time now.
I see you made the upstream fix - well done!  I had started one and was making it way more complicated than your solution ended up being.  Will be very glad to pull in the new kotlinpoet version.  If you're up for another PR for the shouldCompile assertions I'd be glad to have those, too.


registry: remove units from etcd registry upon DestroyUnit()

So far each command "fleetctl destroy unit" has removed job entries from
the etcd registry, under /_coreos.com/fleet/job. But it has not removed
its unit file, under /_coreos.com/fleet/unit. As a result, fleet left
lots of garbages in the etcd registry, so users had to manually clean
them up.
So this patch gets unit contents deleted actually from etcd registry
when DestroyUnit() gets called. To avoid potential hash collisions,
it first fetches a list of units from registry, to check there's any
duplicated entry. Only if no duplicated unit is found, fleetd actually
deletes the unit from registry.
Fixes: https://github.com/coreos/fleet/issues/1456
Fixes: https://github.com/coreos/fleet/issues/1290
Reference: https://github.com/coreos/fleet/pull/1291
/cc @kayrus @wuqixuan

@dongsupark is a functional test possible for this case ?

@tixxdz First of all, I don't think it's possible to test such an extreme case that 2 arbitrary units end up having the same sha1 hash. Theoretically it would be possible, but how would it be practically look like...?
But it's probably possible to write a basic test for submitting & destroying multiple units, to verify that a correct unit has been actually removed. functional/unit_action_test.go doesn't seem to have such tests.

@dongsupark yes of course! yeh I was referencing "verify if the unit was actually removed" parts, thanks!

Creating units with the same hash is easy. echo blabla > foo.service; cp
foo.service bar.service; fleetctl submit foo.service bar.service
On Fri, Mar 18, 2016 at 3:14 PM Dongsu Park<EMAIL_ADDRESS>wrote:

@tixxdz https://github.com/tixxdz First of all, I don't think it's
possible to test such an extreme case that 2 arbitrary units end up having
the same sha1 hash. Theoretically it would be possible, but how would it be
practically look like...?
But it's probably possible to write a basic test for submitting &
destroying multiple units, to verify that a correct unit has been actually
removed. functional/unit_action_test.go doesn't seem to have such tests.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/coreos/fleet/pull/1510#issuecomment-198378257

@jonboulle thx!

@tixxdz

is a functional test possible for this case ?

I added a new functional test TestUnitDestroyFromRegistry, which does not work correctly. On semaphore, etcdctl ls /_coreos.com/fleet/job/hello.service/object fails, because there's no such file. Funny enough, everything works fine during my local test. Not sure what's going on.

So finally I managed to get the new functional test TestUnitDestroyFromRegistry working. It has failed so far because of a wrong keyspace, which should have been /fleet_functional/smoke instead of the default /_coreos.com/fleet Solution was to export cluster.Keyspace() to make it usable from functional tests correctly. (I also fixed other minor bugs, but I'll not get into the detail...)
Though the Semaphoreci still does not succeed completely. Other tests like TestNodeShutdown fail randomly, but I think that has nothing to do with my test. That's probably because of the on-going maintenance work from Semaphore.

@antrik ok, I changed minor things as you suggested, and force-pushed.
BTW after rebasing onto the current master, fleet suddenly stopped to build, because commit 64ec3a433c7de606fe46732837b9f66ec5b2cfc0 removed HashFromHexString. So I had to resurrect the function in registry/job.go. Now it's working.

@dongsupark function was restored. that was temporarily behavior. sorry for confusing.

@kayrus Thanks! :-)
I just removed the temporary commit for HashFromHexString to use again unit.HashFromHexString like before, and rebased it onto master.


mock bls in state tests

mock bls sign and verify in state tests
reduces time from > 1 minute to ~15 seconds
The timing logging in test_full_pos.py becomes meaningless

@hwwhww This isn't done, but I could use some input on the approach

pytest-mock approach looks good and clean! I pulled this PR and tried, 15 sec too 🚀

FYI, #49 may fix the CI.


Rework and simplify selection in TermControl

This commit rewrites selection handling at the TermControl layer.
Previously, we were keeping track of a number of redundant variables
that were easy to get out of sync.
The new selection model is as follows:

A single left click will always begin a pending selection operation
A single left click will always clear a selection (#4477)
A double left click will always begin a word selection
A triple left click will always begin a line selection
A selection will only truly start when the cursor moves a quarter of
the smallest dimension of a cell (usually its width) in any direction
This eliminates the selection of a single cell on one click.
(#4282, #5082)
We now keep track of whether the selection has been "copied", or
"updated" since it was last copied. If an endpoint moves, it is
updated. For copy-on-select, it is only copied if it's updated.
(#4740)

Because of this, we can stop tracking the position of the focus-raising
click, and whether it was part of click-drag operation. All clicks can
become part of a click-drag operation if the user drags.
We can also eliminate the special handling of single cell selection at
the TerminalCore layer: since TermControl determines when to begin a
selection, TerminalCore no longer needs to know whether copy on select
is enabled or whether the user has started and then backtracked over a
single cell. This is now implicit in TermControl.
Fixes #5082; Fixes #4477

A single left click will always clear a selection (#4477)

This one is a bit concerning. Really only because ConHost does not do this. I hate to say it, but it makes me wonder if we should make this particular setting a pointer binding in #1553

A selection will only truly start when the cursor moves a quarter of
the smallest dimension of a cell (usually its width) in any direction
This eliminates the selection of a single cell on one click.
(#4282, #5082)
We now keep track of whether the selection has been "copied", or
"updated" since it was last copied. If an endpoint moves, it is
updated. For copy-on-select, it is only copied if it's updated.
(#4740)
[Paraphrasing:] we can stop tracking the position of the focus-raising
click, and whether it was part of click-drag operation.
We can also eliminate the special handling of single cell selection at
the TerminalCore layer.

These are all things I really like and want. I think the only reason we needed copyOnSelect in TermCore was for the single cell selection. But if we can differentiate between a "tap" pending a selection action vs a "drag" actually selecting, then this model looks a lot cleaner.
Side note: #4082 is probably not the issue you want to link here?

Yeah, I meant #5082. Let’s talk about selection clearing: why do we want to keep the selection when you click somewhere else? All other text buffer applications clear it to start another selection. Right now there’s no way to even abort a selection with the mouse (see complaints in 4477)

I legit think “because conhost did it” is an okay reason for some things, but Terminal also presents a chance to change the things conhost did that don’t make sense. Right now, conhost always starts and completes a selection on a single click so there isn’t a clear state—all you can do is move to another single cell selection, which we’ve established isn’t useful :smile:
If people are using single cell selection to pause output, that’s a failing on our part for us having not provided any other way to pause.

I legit think “because conhost did it” is an okay reason for some things, but Terminal also presents a chance to change the things conhost did that don’t make sense. Right now, conhost always starts and completes a selection on a single click so there isn’t a clear state—all you can do is move to another single cell selection, which we’ve established isn’t useful 😄
If people are using single cell selection to pause output, that’s a failing on our part for us having not provided any other way to pause.

I agree and I think this updated behavior is more correct. Rather, my only concern is that we'll have a number of users coming from ConHost expecting it to work somewhat similarly. That said, we could do what we've done with other features and introduce this new behavior as the default. Then if somebody submits an issue expecting it to work the other way, we then address it by adding it in as a customization option.
I genuinely think what you are proposing is the right behavior. I just don't want people familiar with Windows-isms to suddenly get confused. Even though I think this is one of those bad Windows-isms.

Also, this diff is +35 -120 :smile: you know how much I love deleting code

@leonMSFT after playing with it, any strong feelings on whether focusing a window with a selection should preserve or destroy the selection? Notepad/word destroy it; xterm destroys it; putty destroys it

I think it's fine to clear selection when focusing, makes the click-to-clear selection behavior more consistent 👍

@DHowett-MSFT hey so I'm stoked you implemented my idea so thanks!
When will this make its way into a release? Do I just wait for the app to update from the store or what? Sorry I've literally used the MS store only once before this! :-)

@aybiss thanks! So, we're trying to release approximately monthly, and since this is a behavior change we're probably going to hold it until the next preview build when we change the version number. It should not be too long :smile:


powerdns::backends::mysql should set gmysql-host

db_host is only being using in the mysql::db resource here. https://github.com/sensson/puppet-powerdns/blob/develop/manifests/backends/mysql.pp#L58
I think the following is missing.
powerdns::config { 'gmysql-host':
    ensure  => present,
    setting => 'gmysql-host',
    value   => $::powerdns::db_host,
    type    => 'authoritative',
  }

We should then be able to close https://github.com/sensson/puppet-powerdns/issues/19?

I agree. Together with the $backend_install parameter it should be enough to close #19.

@alexjfisher This is a breaking change for those who already have their database configured elsewhere as it would overwrite gmysql-host. This should only happen for those who are using develop as on other environments having powerdns::db_host set was already required.
I'll add this today.

For some reason my branch included too many commits. I've removed the branch and I'll start over.


Testing cla-bot

let me know if you need another test subject 👀

Thanks, James! Unfortunately I need one who hasn't signed a CLA!

Do we need to/want to disable this for the package list repository? It's showing up there (although it doesn't actually prevent merges - not sure if it would/should for other repos).
https://github.com/SwiftPackageIndex/PackageList/pull/3343

Yes, this is no good. I don't see anything obvious for "all repositories except ..." in the documentation, so I'll disable it all for now.
The alternative is to manually enable it for every repo apart from PackageList, but that's no good as every time we add someone, we'd need to update all repository's allow lists.

Is there a way to whitelist the GitHub Actions bot?

Equally, we don't use it but there is an ability to specify the account used to create the automated PackageList PRs. We could update it to use an account which has signed the CLA (be it you or Sven)?
Just throwing ideas out.

That works great. Thank you James.

Teamwork!


Automatically update README.md towards Docker Hub

It would be nice to synchronize the documentation from your README.md file available on GitHub towards Docker Hub, automatically.
There is a GitHub Action available for this task: here.
I see you are using GitHub Actions already, although not for building images as far as I understand. In case you want to build images automatically on every push to the main branch, synchronizing the README.md might remove a manual task.
Let me know if you would like me to provide a PR for an initial draft of the implementation (both automatic build and documentation synchronization).
Thank you for the great software you have built, I use it basically everyday

Thanks for the offer, but I think the current setup is fine. Having automated builds is not really necessary, if there is something to release I usually follow up on that. Having some blurb on Docker Hub, and a link to the repo would be nice, but syncing the README is a bit overblown - most people engage through Github anyway.


Memory issues in Mac

Which version of the AzCopy was used?
Note: The version is visible when running AzCopy without any argument
I coudn't find the version but I'm using the latest version of Microsoft Azure Storage Explorer with AzCopy enabled.

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.
What problem was encountered?
When I start copying a folder which size is around 80gb, It start taking all the available memory and then my mac shut down.
How can we reproduce the problem in the simplest way?
Try to copy a big size folder in a Mac?
Have you found a mitigation/solution?
No
Attachments

Currently, AzCopy sets its memory usage target at 0.5 GB per CPU, plus some overhead.  How many CPUs does your Mac have?
BTW, we are looking at moving to directly detecting the amount of RAM instead of basing our cap on CPU count.

I'm not sure if it's the answer you want but I have a Inter i7 2.9  Quadcore. Thank you for your help.

Hmm, if it's quad core, that definitely suggests something is wrong with the way AzCopy is acting, because it should not use 10 GB on a 4 core machine.
One last question, just to make sure we try to reproduce this in the right way: roughly what was the typical size of the files in the folder you were transferring.  E.g. a few hundred KB each, a few MB each, several GB each?

I'm transferring around 8M of JPG pictures, each one has a size of around ~9Kb.
Also, there's any work around to set the memory usage manually?

Thanks. That will be useful information as we try to reproduce it.  Currently there is no workaround to set memory usage manually. We are considering that feature for a future release.
In the meantime, for a workaround, can you use Storage Explorer with AzCopy disabled as a workaround. (that should work for everything except Data Lake Gen2, for which it can't be disabled).

Hi @focux, thanks a lot for reaching out! I'm working on reproducing your scenario. For my information, were you uploading or downloading the 8 million images? And how were the images organized? Was there a particular folder structure? Were some images in deeper paths from the root? For example: root/.../.../.../.../.../.../..../example.jpg.
@JohnRusk I'm suspecting that the list-of-files flag might be the culprit. I'll try to repro with an accurate setup.

@zezha-msft Oh yeah, that's a very plausible theory. (And we have no alternative theories that explain it!)

@focux We are pretty certain that list-of-files is the culprit.  Ze has re-written that feature to avoid the problem, and its scheduled for release in 10.3

Awesome! Thank you. I'm going to close the issue.


[catch2] build static lib on WIN32

Fix #26880
[catch2] enables dynamic builds after version 3.10, but on windows it fails to provide exported symbols.
Error:
1>------ Build started: Project: tests, Configuration: Debug x64 ------
1>tests.cpp
1>tests.obj : error LNK2001: unresolved external symbol "public: static int Catch::StringMaker<double,void>::precision" (?precision@?$StringMaker@NX@Catch@@2HA)
1>C:\ubuntu\temp\msvc-sln\Debug\tests.exe : fatal error LNK1120: 1 unresolved externals
1>Done building project "tests.vcxproj" -- FAILED.

Why patch instead of vcpkg_check_linkage?

@Osyotr I don't want to break the other triplets.

You won't break other triplets.

@Osyotr

Why patch instead of vcpkg_check_linkage?

Best way, thanks for the reminder. :)


[CSL-116] VssCertificate with TTL

@georgeee
TBD after CSL-19
Idea is to introduce some TTL for VssCertificate

@gromak
I asked some questions about VssCertificates in #paper-pos. Let's wait until they are answered.

@gromak
I was told that the way we deal with VssCertificates now is fine.
Of course TTL should be added. It's important not only one may want his VssPublicKey to expire after some from the security point of view, but also because w/o TTL set of all active certificates may become very big eventually. I suppose it can be solved by two limits: limit on number of certificates in block and limit on certificate TTL. This way if we know that certificate is valid for at most k slots and each block has at most n certificates, then we will have to operate on at most k * n certificates and we can ensure it fits in RAM (or at least that it is not extremely big set). It makes sense to prioritize certificates from nodes with bigger stake (i. e. if we have more than n certificates and need to create block, we choose certificates from nodes with bigger stake).
Another related thing is certificate revocation. I think it should be supported too. For example, if VSS secret key is compromised, it makes sense to revoke certificate for this key. I think it shouldn't be hard to do.

@gromak
I suppose it shouldn't be Postponed anymore. W/o TTL set of all active certificates will become too big eventually.


[web]: Build doesn't recognize diferent imports for the same file

Hi, Trying to build the app for deployment using the flutter build web command on version 1.12.5 generates the following error:

Entire StackTrace:
#0      Dart2JSTarget.build (package:flutter_tools/src/build_system/targets/web.dart:156:7)

#1      _BuildInstance._invokeInternal (package:flutter_tools/src/build_system/build_system.dart:526:25)

#2      _BuildInstance.invokeTarget.
(package:flutter_tools/src/build_system/build_system.dart:481:35)
#3      new Future.sync (dart:async/future.dart:224:31)
#4      AsyncMemoizer.runOnce (package:async/src/async_memoizer.dart:43:45)
#5      _BuildInstance.invokeTarget (package:flutter_tools/src/build_system/build_system.dart:481:21)

#6      BuildSystem.build (package:flutter_tools/src/build_system/build_system.dart:419:36)
#7      buildWeb (package:flutter_tools/src/web/compile.dart:41:50)

#8      BuildWebCommand.runCommand (package:flutter_tools/src/commands/build_web.dart:57:11)
#9      FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:607:18)
#10     _asyncThenWrapperHelper. (dart:async-patch/async_patch.dart:73:64)
#11     _rootRunUnary (dart:async/zone.dart:1134:38)
#12     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
#13     _FutureListener.handleValue (dart:async/future_impl.dart:139:18)
#14     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:680:45)
#15     Future._propagateToListeners (dart:async/future_impl.dart:709:32)
#16     Future._completeWithValue (dart:async/future_impl.dart:524:5)
#17     Future._asyncComplete. (dart:async/future_impl.dart:554:7)
#18     _rootRun (dart:async/zone.dart:1126:13)
#19     _CustomZone.run (dart:async/zone.dart:1023:19)
#20     _CustomZone.runGuarded (dart:async/zone.dart:925:7)
#21     _CustomZone.bindCallbackGuarded. (dart:async/zone.dart:965:23)
#22     _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
#23     _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)
#24     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#25     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:175:5)
Exception: lib/main.dart:46:31:
Error: The argument type 'List<Tema/1/>' can't be assigned to the parameter type 'List<Tema/2/>'.
'List' is from 'dart:core'.
'Tema/1/' is from 'lib/main.dart'.
'Tema/2/' is from 'package:appreport/main.dart' ('lib/main.dart').
ConnectivityBloc(temas: temas, navigatorKey: _navigatorKey);
^
Error: Compilation failed.
I believe this is because in one file i have "package:appreport/main.dart" as an import. Importing the file with its relative path("../main.dart") fixes the problem. It seams dart2js can't identify the different imports methods are referencing the same file.
P.S. The code i'm running works perfectly on flutter run -d chrome

Duplicate of https://github.com/flutter/flutter/issues/43846


imagepicker 样式和api无效

[ ] I'm sure this does not appear in the issue list of the repository

Basic Info

Package Name And Version<EMAIL_ADDRESS>Phone Model And Version: 移动端h5


SAML based SSO

Fixes #4974
Describe the changes you have made in this PR -

added gem ruby-saml
fix devise_saml_authenticable and ruby-saml gem versions compatibility
added saml_update_resource_hook (this resolves overriding  issue of create method in saml_session_controller by devise_authenticable_gem  )
add methods in saml_session_controller for handling SAML request
added Routes for controller underuserscope
updated the UI

Screenshots of the changes (If any) -
https://github.com/CircuitVerse/CircuitVerse/assets/96644946/92754ccc-a7ce-4acd-8423-dc2ab36b6253
Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

@Asrani-Aman I  checked the video and basic implementation looks fine so far. Please address the review comments. In the ui I see sign-on button labled as Single Sign-on .. Its a bit misleading may be we can rename it to Single Sign on with Okta ( I suppose thats what we are supporting as of now ?). Also please commit Gemfile.lock and one sample devise config for Single sign on configurations

@tachyons @smritigarg
Good evening mentors,
I've removed the ruby-saml gem, and everything is now working smoothly without it.
and also as you suggest  I don't need to explicitly define the methods in saml_session_controller.rb as it might not handle all the cases and Instead, using the implicit Devise methods works well. The only thing we need to add is Devise.saml_update_resource_hook; otherwise, it will cause errors as mentioned in attached video in PR

@smritigarg
I tested it with Okta, and I believe it will work with other IDPs as well. We just need to add the respective ['IDP_SSO_URL'] and ['IDP_FINGERPRINT'] of each Identity Provider in the .env file.
Could you please provide more context regarding adding the Devise config for Single Sign-On configurations?
Do you mean adding documentation?

@smritigarg done Changes

Could you add documentation for how to configure Circuitverse with SAML using Okta and keycloak ?

Also update Pr description

@tachyons @smritigarg

Added the Documentation for or how to configure Circuitverse with SAML using Okta and keycloak

May you please give a look at it PR Link


I can't start Ubuntu from my Chromebook.

I put in sudo startunity in the crosh shell. I get some weird error with a bunch of (EE) though. I'm not sure how to right click or copy from the crosh shell, as it won't let me, but if you can tell me i'll be able to post exactly what it says. Thanks.

you could try to update it if you have recently updated chrome os
sudo sh -e ~/Downloads/crouton -u -n trusty

pleas post the output of the commands

instead of using the crosh shell do this
(ctrl+alt+t) type shell then enter sudo startunity
highlight the output then use keyboard shortcut (ctrl+shift+c)

its ok i fixed it. it turns out i accidentally got out of developer mode.
On Tue, Jul 14, 2015 at 11:11 AM, giliammc<EMAIL_ADDRESS>wrote:

instead of using the crosh shell do this
(ctrl+alt+t) type shell then enter sudo startunity
highlight the output then use keyboard shortcut (ctrl+shift+c)
—
Reply to this email directly or view it on GitHub
https://github.com/dnschneid/crouton/issues/1917#issuecomment-121277556.

great!
you should close the issue

how do i do that?
On Thu, Jul 16, 2015 at 4:46 PM, giliammc<EMAIL_ADDRESS>wrote:

great!
you should close the issue
—
Reply to this email directly or view it on GitHub
https://github.com/dnschneid/crouton/issues/1917#issuecomment-122087932.

Push the comment and close button
On Sat, Jul 18, 2015, 3:39pm TheGlamourGroup<EMAIL_ADDRESS>wrote:

how do i do that?
On Thu, Jul 16, 2015 at 4:46 PM, giliammc<EMAIL_ADDRESS>wrote:

great!
you should close the issue
—
Reply to this email directly or view it on GitHub
<https://github.com/dnschneid/crouton/issues/1917#issuecomment-122087932
.

—
Reply to this email directly or view it on GitHub
https://github.com/dnschneid/crouton/issues/1917#issuecomment-122595812.


feat: removeFlowFileOnStartup option

Closes https://github.com/influxdata/data-acquisition/issues/158
Adds new property to NifiClusterSpec: removeFlowFileOnStartup which defaults to true and matches existing behavior. If set to false, the flow.xml.gz file will not be deleted during the node's startup.

I think you’re missing a few file changes. My list last night was clearly incomplete, sorry. One of these appears to be auto-generated. See the other PR where we’ve done this: https://github.com/influxdata/nifikop/pull/11/files


fix: update interface path and ESLint.

please review.

Hi, @downdawn
My local execution of eslint seems to format all the files.
IDE: WebStorm

In theory, if the .eslintrc.js configuration is the same, it shouldn't change.

I will merge it and pull it again.


Implement various measures of tortuosity

Tortuosity has been defined in many different ways in the literature:
From https://www.europeanhealthjournal.com/2017/02/14/skeleton-based-cerebrovascular-quantitative-analysis/ :

In another study, Jens et al. studied abnormal extracranial ICAs and used lateral cervical displacement to determine tortuosity [16]. Studies by Grisan et al. and Shelton et al. concerning vessel tortuosity measurements used retinal blood vessels to calculate the ratio of arc length to chord length and curvature [17, 18].

The materials and methods contain various other approaches fitting b-splines. Would be nice to have those implemented in skan.
CC @marlene09

Hi @jni,
I am not sure what you wish, but my software that relies on skan fits B-splines on the skeleton network. Maybe you can extract useful bits out of it.

Thanks @CsatiZoltan!


Could not download Plugin

Lapce Version
Lapce 0.2.5
System information
Mac OS 13.2 (22D49)
Describe the bug
I cannot either upgrade plugins or download new plugins.
Additional information

I tried to change the network environment, but the problem occurred still.

Seems like DB de-synced again and our API is borked.

Sorry to add to your troubles, this is not urgent and I can continue to use the old version.
So I don't need to do anything else now, just wait for you to fix the API?

It's fixed now but the problem will happen again until I figure out what's the actual problem.
Will need to ask @dzhou121 how it was configured/works since I did not deploy plugins.lapce.dev.

Many thanks for your fixing, I just checked thee function which can work properly now. And I upgraded my needed plugins successfully. Sorry for adding your troubles and thanks again.

still this same issue on all versions of lapce, nightly, built from source and the packaged Arch Linux version.
distro is Arch for all, using NetworkManager
i've also tested it on a NixOS vm and it has the same issue
on the nightly version all plugins change to say "Installing" but stay that way forever and never install, on the packaged Arch Linux version it does the same as in OP's image
had to clone plugins to the folder and build them myself to get any plugins installed.
couldn't find any output showing any errors unfortunately

been messing with the api and it appears the download from cloudflare storage returns Unauthorized
e.g: https://plugins.lapce.dev/api/v1/plugins/dzhou121/lapce-rust/0.2.4/download
the cloudflare cdn storage link this api request returns gives a 401 with an XML error 'Unauthorized'
I inserted some log prints, and confirmed that 401 is what it is.
so it fails here, and this error message is never propagated to any sort of logging mechanism, or the ui resulting in an infinite "Installing" message:
https://github.com/lapce/lapce/blob/980b025886bb52c151cd8954c7547fd94be35c73/lapce-proxy/src/plugin/mod.rs#L1278C27-L1278C27


Intial documentation

This is just a start. I made some assumption about how the utility is going to work. Feel free to let me know if I should correct them. (Or you can merge and then rewrite as you see fit.)
I also made a start on documenting the Enigma Xml format. Ultimately, I don't think md is the best tool for it. I wonder what tool the MusicXml group used, because it is quite good. I'd love to see us migrate to that at some point.
I know you said you don't think the documentation of the format is in scope for this repo. But I hope you will let us park it here until a better place comes along.
Resolves #2.

I think a better approach is to create a separate repo for developing an xsd that in turn can be used to create a doc site. If you don't want to park the enigmaxml doc file here, that's fine. I still think the readme info may belong here.

Closing this and moving to a different repo. You might want to provide a link to it from your README. Will post the link to Discord.

Okay! That is perfectly fine.  I would like to keep a lot of the more general stuff that you had mentioned though.  I think a road map to what's zipped together and the basics that the score.dat file contains the bulk of the data, and that it is what we call enigmaXML is appropriate.  Then I would reference your repo or the lua repo.


The NodeJS sample is not working

I am trying to make this work on a windows laptop. After cloning the stage branch and trying to execute the command node app.js after doing npm install, I am getting the below error
(node:11608) UnhandledPromiseRejectionWarning: Error: error:0909006C:PEM routines:get_name:no start line
at Sign.sign (internal/crypto/sig.js:80:26)
at Object.sign (C:\SUVA_DATA\centralized-business-services\adobeio-auth\JWT\samples\adobe-jwt-node\node_modules\jwa\index.js:152:45)
at Object.jwsSign [as sign] (C:\SUVA_DATA\centralized-business-services\adobeio-auth\JWT\samples\adobe-jwt-node\node_modules\jws\lib\sign-stream.js:32:24)
at Object.module.exports [as sign] (C:\SUVA_DATA\centralized-business-services\adobeio-auth\JWT\samples\adobe-jwt-node\node_modules\jsonwebtoken\sign.js:204:16)
at authorize (C:\SUVA_DATA\centralized-business-services\adobeio-auth\JWT\samples\adobe-jwt-node\node_modules@adobe\jwt-auth\index.js:63:17)
at Object. (C:\SUVA_DATA\centralized-business-services\adobeio-auth\JWT\samples\adobe-jwt-node\app.js:6:5)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
(node:11608) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:11608) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Can you please throw some light on the same.

Hi @suvadeepdey,
I have validated the NodeJS sample and able to generate the access token. Looking at the error it seems the private key certificate isn't correct. Let me know if you are still facing the issue.


If you go to edit profile data, it's usually blank until you refresh the page

Probably a wontfix for this weekend, but documenting regardless

Ah, I think this is what I found here while fixing #664.


fix: tutorial on mobile

Create CSS based navigation with toggle button for smaller screens
Fix styles that were preventing the layout to correctly resize on smaller screens
Bonus: updated nav and body background colors so they match the colors on our builtin docs

Before
After (Closed)
After (Open)

Test failures look unrelated.