<FromGitter>
<meltheadorable> who knows how many functions
<FromGitter>
<dscottboggs_gitlab> also, you only need a subset of one version of Z39.50, as opposed to supporting all features of all of the versions
<FromGitter>
<meltheadorable> yeah
<FromGitter>
<meltheadorable> i just need what’s necessary to query modern library catalogs
<FromGitter>
<dscottboggs_gitlab> That said...it would probably take not much time at all to just bind to yaz and use only a few functions rather than reimplementing all that, and then after you've worked out exactly what you'll need you can refactor from there to replace the C functions with Crystal code one by one.
<FromGitter>
<meltheadorable> the goal is basically just to stand up a modern JSON API in front of this Z39.50 client so I can use it for all my apps that need to query book metadata
<FromGitter>
<meltheadorable> binding yaz would require learning yaz, id rather learn the protocol ¯\_(ツ)_/¯
<FromGitter>
<bajro17> @watzon in this git repo cadmium in file trie.cr is that radix tree?
<FromGitter>
<dscottboggs_gitlab> that's fair
<FromGitter>
<watzon> Yes it is @bajro17
<FromGitter>
<watzon> I actually didn't write that one, it was a contribution
<FromGitter>
<dscottboggs_gitlab> you could just use the TCP::Socket and write text to it of course
<FromGitter>
<bajro17> Most beautiful implementation I love it
<FromGitter>
<meltheadorable> There is just too much there i don’t need
<FromGitter>
<dscottboggs_gitlab> yeah you only need that top-left-middle block
<FromGitter>
<meltheadorable> & i dont really want to provide super incomplete bindings to such a massive library
<FromGitter>
<meltheadorable> basically im just under the impression that the protocol is probably simpler than the library, and if i have a choice between finding and deciphering the part of the library i need vs. learning the protocol and writing native crystal to do it, the native crystal feels better
<FromGitter>
<meltheadorable> i could get halfway through the protocol standard docs and realize i made a mistake and switch courses
<FromGitter>
<watzon> The protocol sounds like a pain in the ass from the Wikipedia entry
<FromGitter>
<meltheadorable> yeah it does
<FromGitter>
<meltheadorable> but im also planning to write SMTP and IMAP servers so im a glutton for punishment
<FromGitter>
<watzon> Lol you sure are
<FromGitter>
<watzon> But that would be cool to have in Crystal
<FromGitter>
<dscottboggs_gitlab> fuck yeah it would
<FromGitter>
<dscottboggs_gitlab> why don't you start with one of those since they'll be more well documented?
<FromGitter>
<meltheadorable> it would be SO cool to have in crystal — and the end goal is even cooler — a totally modern mail server stack with a programmatic interface
<FromGitter>
<meltheadorable> no more dumping mail into random shitty folders on the filesystem
<FromGitter>
<meltheadorable> no more esoteric dovecot configs
<FromGitter>
<meltheadorable> i will probably do the mail server first fwiw, i don’t need z39.50 for a while, and there’s an existing ruby gem that binds yaz i could use for a prototype for my JSON api if need be
<FromGitter>
<meltheadorable> i’d just rather do native socket stuff in crystal than ruby
<FromGitter>
<meltheadorable> if i were gonna reimplement
<FromGitter>
<meltheadorable> but its a totally avoidable task using other tools if i just wanna get a prototype going
<FromGitter>
<watzon> Hell yes please
<FromGitter>
<meltheadorable> SMTP is more important than imap because i might just write a web app to query the SMTP server’s JSON API
<FromGitter>
<zenithlight> the json rpc module exposes a blocking api so i'm not sure how i would support that if i wrapped the non-blocking WS api, but i'm new to the lang so maybe there's something i haven't considered
gangstacat has quit [Ping timeout: 252 seconds]
gangstacat has joined #crystal-lang
<FromGitter>
<watzon> Speaking of Cadmium watzon/cadmium#19
<FromGitter>
<watzon> Nice to be getting contributions
<FromGitter>
<watzon> @zenithlight unfortunately it doesn't look like there's been any movement on that
<FromGitter>
<watzon> But it looks like someone could open a PR and get it accepted
<FromGitter>
<zenithlight> i could certainly try to do that
<FromGitter>
<zenithlight> it seems like there was some contention on the design/implementation though
<FromGitter>
<watzon> Yeah that seems to always be the cased
teardown has quit [Ping timeout: 245 seconds]
DTZUZU has quit [Quit: WeeChat 2.2]
DTZUZU has joined #crystal-lang
<FromGitter>
<rvprasad> While stepping thru the a crystal program using gdb, I am observing the following stack trace.
laaron has joined #crystal-lang
DTZUZU has quit [Quit: WeeChat 2.2]
DTZUZU has joined #crystal-lang
<FromGitter>
<meltheadorable> A;sldkjfa;ldskjfa async stuff is confusing send help
<FromGitter>
<watzon> Yes. Yes it is.
<FromGitter>
<meltheadorable> especially because im pretty sure the code im using is already doing its own stuff with fibers underneath and i can’t tell what is/isn’t blocking, i need to spawn both a client and a server in my tests struggling to figure out how to handshake their communication enough to write any expectations
teardown has joined #crystal-lang
<FromGitter>
<meltheadorable> effffffffffff
<FromGitter>
<meltheadorable> i have a headache
<FromGitter>
<tenebrousedge> are you logging the everything in your threads?
<FromGitter>
<meltheadorable> i just deleted everything i had tbh
<FromGitter>
<meltheadorable> and re-reading the concurrency chapter in the gitbook
<FromGitter>
<meltheadorable> but mostly i don’t really know what kind of API would be appropriate for this yet and the complexity of trying things out is discouraging experimentation 😂
teardown has joined #crystal-lang
<FromGitter>
<tenebrousedge> what are you trying to do?
<FromGitter>
<meltheadorable> trying to write a test that asserts my SMTP server will accept a connection — i need to spawn both the server and the client in the test and then somehow coordinate the fibers talking to each other in order
laaron has joined #crystal-lang
<FromGitter>
<meltheadorable> the server is “dumb” i mostly want the client to drive, because the tests should mostly be that the client is able to send a specific message and get a specific response, at least at first
<FromGitter>
<meltheadorable> but i’m not sure how to appropriately hand control back and forth
<FromGitter>
<tenebrousedge> that sounds like you're testing either OS-provided functionality or stdlib-provided functionality
<FromGitter>
<watzon> Sounds like you need to start the server as a background process and handle most of the testing with the client
<FromGitter>
<meltheadorable> no i’m writing an SMTP server, i definitely am testing that
<FromGitter>
<tenebrousedge> and you can't use dummy I/O?
<FromGitter>
<meltheadorable> not really?
<FromGitter>
<tenebrousedge> why not?
<FromGitter>
<meltheadorable> because then i can’t test the SMTP server
<FromGitter>
<meltheadorable> if i mock the server i’m mocking the thing i’m trying to test, and i can’t mock the client, the server doesn’t have a public interface beyond what it sends over the socket
<FromGitter>
<meltheadorable> and thats dictated by the protocol
<FromGitter>
<meltheadorable> not user code
<FromGitter>
<meltheadorable> so i’m not sure how you’d have me dummy it up
<FromGitter>
<meltheadorable> @watzon yeah that might work, but i’m not sure how to do that ^_^;;;;;;;
<FromGitter>
<meltheadorable> at the very least, the client/server interface i need to test with the client driving a running server process — once the server gets messages i’ll also need ways to decide what to do with those, but it’s easier to reason around the client/server interface first
<FromGitter>
<watzon> You'd probably have to make use of `Process.fork`
<FromGitter>
<watzon> Idk though, I haven't tried anything like that before
<FromGitter>
<meltheadorable> btw the client in this case is just a raw TCPSocket sending appropriate messages
<FromGitter>
<meltheadorable> SSL is a problem for future me
<FromGitter>
<tenebrousedge> I mean, Crystal has socket and server tests that don't use `Process.fork`, yeah?
<FromGitter>
<meltheadorable> suppose it probably should
<FromGitter>
<watzon> And then just use your TCPSocket to send messages back and forth
<FromGitter>
<tenebrousedge> I don't think that starting another process should be necessary here
<FromGitter>
<watzon> You're probably right actually
<FromGitter>
<watzon> You should be able to just start it in a thread
<FromGitter>
<meltheadorable> the TCPServer specs don’t test actual connections as best as i can tell, presumably because unlike an SMTP server a TCP Server doesn’t have a protocol it’s conforming to
<FromGitter>
<watzon> Hmmm
<FromGitter>
<tenebrousedge> isn't HTTP a protocol layered on TCP?
laaron has quit [Remote host closed the connection]
laaron has joined #crystal-lang
teardown has quit [Ping timeout: 245 seconds]
alex`` has joined #crystal-lang
laaron has quit [Remote host closed the connection]
laaron has joined #crystal-lang
ht__ has joined #crystal-lang
ht_ has quit [Remote host closed the connection]
ht__ is now known as ht_
sorcus has joined #crystal-lang
sagax has joined #crystal-lang
<FromGitter>
<dubesoftware> Greetings everyone. What’s the current state of parallelism in Crystal?
<FromGitter>
<straight-shoota> @dubesoftware It's in the works
<FromGitter>
<dubesoftware> Thanks @straight-shoota . I’m coming from Python (stopped at Ruby and liked it but need more horsepower for large concurrent apps).
<FromGitter>
<dubesoftware> I like what I’m seeing with Crystal. Why does Crystal execute in a single thread? Isn’t that leaving money on the table in an age of multi-core computing?
<FromGitter>
<j8r> @dubesoftware if you need large scaled computing, you multi-threading alone won't help
<FromGitter>
<j8r> you can do multi process, then no limit of scaling them on multiple machines
<FromGitter>
<j8r> e.g. look at Node.js
<FromGitter>
<dubesoftware> @j8r is something like node suited to CPU-bound operations?
<FromGitter>
<dubesoftware> In my view strong concurrency is fine for things like network I/O, but for CPU-intensive work e.g. number-crunching, using all cores matters?
dingenskirchen has quit [Ping timeout: 245 seconds]
<FromGitter>
<j8r> @dubesoftware what I mean is you can do multi process to use all the cores
<FromGitter>
<j8r> the program would be horizontally scalable, instead of only vertically
<FromGitter>
<dubesoftware> Gotcha thanks @j8r
<allan>
@j8r don’t think you are right there: I can max out all my CPU cores with a single process (using multiple OS threads)
<FromGitter>
<dubesoftware> Kudos to all contributors, the documentation for Crystal is very concise. I’ll join in as I pick up speed and help get things done.
<FromGitter>
<j8r> allan: not in Crystal for now, not multithreaded
<FromGitter>
<j8r> you can fork, through
<allan>
@j8r right, Crystal does not use OS threads
<FromGitter>
<dubesoftware> @FromIRC you explained much better what I was trying to get to
<allan>
@j8r but fork is much more expensive and also have data sharing issues
<FromGitter>
<j8r> yes :|
<FromGitter>
<j8r> TD;DR: I mainly see the multi-core issue for games
<FromGitter>
<j8r> and multimedia like FFMPEG
<allan>
there is *lots* of processing tasks that can gain from (cheap) concurrency; right now Crystal lack that
<allan>
basically anytime you run the same operation on a large array
<FromGitter>
<dubesoftware> @j8r Is it not an issue cost-wise for any app type? If I have to throw on more hardware to scle horizontally before squeezing out max computing power from what I’m already paying for?
<FromGitter>
<j8r> it's simpler to design an application with multi-process, with network communication, that designing an app with multi-core + multi-process
<FromGitter>
<j8r> Multi-threading has a cost too. A 2 threaded program won't be 2x faster, maybe 1.5x
<allan>
@j8r it depends on the abstractions offered by the language/environment and the task to be executed
<FromGitter>
<kniknoo> That's what I was thinking, a realy tiny program my get slowed down just from the time to generate the thread.
<allan>
I have 4 cores in my machine and I’ve had plenty of tasks improved with almost a factor of 4 when switching to concurrency
<allan>
and for “ease of design” there are many good abstractions over threads
<livcd>
were not the efforts for parallelism mainly driven by ysbaddaden ?
<allan>
I am not aware of anything that simplifies fork because then you deal with two OS tasks that have separate memory spaces etc. and would have to communicate over sockets
<allan>
@j8r also, using threads is always optional, if you do not use them, you do not pay for them, so even if what you say is true, it is not an argument for not offering it as an option
<livcd>
I think Ruby will get parallelism before Crystal
dingenskirchen has joined #crystal-lang
<allan>
livcd: you could view the non-parallel nature of ruby’s Thread class as an implementation detail, e.g. jruby and rubinius both use OS threads and thus offer “real” parallelism (well, technically jruby uses JVM threads)
<FromGitter>
<dubesoftware> I was just poring over JRuby docs now.
<allan>
@dubesoftware in my experience, Crystal is 10+ times faster than (non-parallel) ruby, so if you are after performance, Crystal is probably still a better choice than even jruby/rubinius, but it does sort of suck that it does not support real threads, so if performance beats syntax, you should probably go with neither Crystal nor ruby (depending on your workload of course, i.e. if your work is 100% serial, there would be no performance g
<allan>
ains from parallelism)
<allan>
also, if bottlenecks are network or i/o, the Crystal approach is fine for achieve some level of concurrency
<allan>
speaking of performance: I wrote a simple program in Crystal which takes ~10 seconds to build (using --release): is that (currently) expected?
<allan>
without --release it of course builds much faster, but the executable runs a lot slower
<allan>
as the executable does a lot of work, the --release does make it >10 seconds faster, so definitely I want the --release build, but somewhat annoying that so much of the time is spent during build
<FromGitter>
<ArtLinkov> So... something super weird is happening to me right now... ⏎ For some reason new code changes do not run when I compile my code, it detects changes in the spec files (and runs them) but for some reason it compeletely ignores all the changes in the src code, running and older version of my code ⏎ Has anyone ever had this kind of problem?
<FromGitter>
<ArtLinkov> I tried the basics obviously, like restarting everything and even changing to another IDE
<FromGitter>
<dubesoftware> @FromIRC in the end my biggest concern is around cost. If I run a SaaS app on a cloud platform where I pay for CPU, RAM etcetera, am I right to think that language choice will impact the monthly running costs differently for the same work?
<allan>
@dubesoftware definitely, a scripting language has significant overhead compared to a compiled language (despite producing the same result)
<allan>
but… most people would think about productivity first, and scaling/cost second
<allan>
also, I am not sure how big you intend your SaaS platform to become, but if you have a fixed instance then you just pay for the machine regardless of how much CPU it uses
<allan>
you can get them as low as $5/month, I would definitely start there rather than metered CPU billing
<FromGitter>
<dubesoftware> Food for thought, thanks.
<allan>
that is something you normally do if you have uneven load
<FromGitter>
<j8r> allan but designing a multi core program takes time, looks at AA games
<FromGitter>
<kniknoo> I mean, it takes 5 seconds for me to build --release hello world. That sounds right.
<FromGitter>
<tenebrousedge> no, I think that should be more or less instantaneous
<FromGitter>
<j8r> not in release
<FromGitter>
<meltheadorable> it seems weird to care so much about the compilation time — don’t recompile constantly in release mode?
<FromGitter>
<dubesoftware> 5 secs to compile a ‘hello world’ program is a lot.
<FromGitter>
<meltheadorable> who cares
<allan>
@j8r really depends on what you design; you may have a regular single-threaded app, but then you need to convert markdown to HTML for 100+ files and you do it in a loop, that is slow and _extremely_ easy to do in parallel (when the language has the right abstraction)
<FromGitter>
<kniknoo> It sure is. That's why I wanna account for not caring about load time in my live-code scenario. It'll start when it gets there.
<allan>
@meltheadorable I am testing the executable after making changes to the code, and without --release the executable will run > 10 seconds slower, so I do need to build as --release
<FromGitter>
<meltheadorable> then wait the 10s
<FromGitter>
<tenebrousedge> @ArtLinkov are you sure that the changes are being saved? and are you sure they're being saved to the same place? what command are you using to compile?
<FromGitter>
<kniknoo> Yeah, it's worth the wait in the long run.
<FromGitter>
<meltheadorable> or make it do less work at a time while you’re testing so that you can build it out of release mode
<FromGitter>
<meltheadorable> ¯\\\_(ツ)\_/¯
<FromGitter>
<kniknoo> But I do recognize it as a pain point in some domains which is why I'm so sure that the idea I'm describing of modular crystal binaries tied together by ruby could be useful.
<allan>
@meltheadorable I didn’t ask for advice about how to deal with the delay, I asked if this is delay is to be expected, a simple “yes” would have sufficed
<FromGitter>
<ArtLinkov> @tenebrousedge yes, I even tried changing the code with a regular text editor
<FromGitter>
<meltheadorable> you already got the yes, i was mostly talking to the other person complaining that it should be instantaneous
<FromGitter>
<dubesoftware> Fwiw I don’t think there’s anything ‘weird’ in wanting to figure out a way around a problem
<FromGitter>
<ArtLinkov> I'm just running crystal spec from the terminal, as I always do
<FromGitter>
<dubesoftware> ( ͡° ͜ʖ ͡°) <— every community needs one, and sooner or later they appear.
<FromGitter>
<meltheadorable> if you want instantaneous use a language that does less work, crystal takes time to compile because it’s doing a ton of interesting things to have all the power it does, theres a bunch of simpler languages that put more work on you than on the compiler to use if you need near-instant compilation
<FromGitter>
<kniknoo> Get in line behind the ongoing Abstract Saga...
<FromGitter>
<meltheadorable> there’s no way to come out on the winning side of every possible tradeoff
<FromGitter>
<meltheadorable> crystal trades compile time performance for runtime performance, that seems fair almost all the time
<FromGitter>
<meltheadorable> in most scenarios you will spend more time running your program than compiling it
<FromGitter>
<dubesoftware> Go doesn’t need to make that excuse. Blazing fast compilation and still fast as anything out there on the metal.
<FromGitter>
<meltheadorable> go has no language features
<FromGitter>
<dubesoftware> uh. ok.
<FromGitter>
<meltheadorable> it’s 1/10th as expressive as crystal is, that’s how it manages, but write go if you want
<FromGitter>
<kniknoo> Crystal is linguistically very complex
<FromGitter>
<tenebrousedge> the tradeoff is developer performance. Crystal is faster than interpreted languages, but there's a runtime cost for the language features too
<FromGitter>
<tenebrousedge> but either way, compilation time should be improvable
<FromGitter>
<meltheadorable> you can’t have the kind of macro system and type inference crystal has and compile instantaneously
<FromGitter>
<kniknoo> Yeah, Crystal's just a baby still.
<FromGitter>
<meltheadorable> improvable sure, but it’s never going to compile as quickly as languages designed to have no features so that they can compile quickly
<FromGitter>
<kniknoo> Probably not, worth every lost second of life for me. :D
<FromGitter>
<meltheadorable> go and swift were designed to be painless to compile at the expense of developer productivity
<FromGitter>
<meltheadorable> its fine to like them but crystal is on a very different path
<FromGitter>
<tenebrousedge> @ArtLinkov your code changes show up in `git status` ?
<allan>
swift was also pretty slow to compile initially
<allan>
I am sure crystal will improve
<allan>
all the features themselves does not make this impossible, at least not in any way that I am aware
<livcd>
but Crystal is basically "done" no ? The only remaining parts require a lot of man/brain power
<allan>
I have not studied it in detail, but it would almost seem like when you build your executable, it also builds the standard library with it, otherwise there is no reason “Hello world” would take 5 seconds
<FromGitter>
<ArtLinkov> @tenebrousedge yes
duane has joined #crystal-lang
<FromGitter>
<kniknoo> I wonder how much of a difference strict typing everything would make.
<FromGitter>
<dubesoftware> @meltheadorable this has nothing to do with whether anybody likes language A more than B. When you say things like ‘...but write go if you want’ or throw the ¯\_(ツ)_/¯ emoji because you feel you know something someone didn’t, that comes off as a bit arrogant.
<FromGitter>
<dubesoftware> My first day here and I am surprised to see such behaviour.
<FromGitter>
<dubesoftware> Everyone here is here for various reasons, the common one being that we see something in Crystal and want it to take even more hold and continue to be better. There is no contest being held here.
* allan
agrees with dubesoftware
<FromGitter>
<meltheadorable> maybe on your first day here you should avoid attacking the language for having longer compilation times than Go
<FromGitter>
<tenebrousedge> no one is attacking Crystal
<FromGitter>
<tenebrousedge> clearly we're all interested in using it
<FromGitter>
<dubesoftware> And you double down. Nothing has been attacked. We are having an open discussion about the caveats of having (or not having) certain language features. It would appear to be a bit puerile to turn that into an attack.
<allan>
this document mentions both multithreading and incremental compiles (as on the roadmap)
<FromGitter>
<tenebrousedge> multithreading is under active development
lucasb has joined #crystal-lang
<FromGitter>
<dubesoftware> Yes, as @straight-shoota alluded earlier in response to my very first question. That’s what set off the lively conversation.
<allan>
@tenebrousedge the roadmap mentions “so fibers can run on multiple threads”, do you know if there are new abstractions planned beyond fibers?
<FromGitter>
<dubesoftware> Thanks for the roadmap. It’s great to see the direction things are headed.
<FromGitter>
<tenebrousedge> I don't know, but there's an issue for it on Github
<FromGitter>
<tenebrousedge> @ArtLinkov crystal version / OS ?
<FromGitter>
<tenebrousedge> and if that doesn't work, maybe try creating another project with those contents real quick?
<FromGitter>
<ArtLinkov> @tenebrousedge hmm... new module seems to work
<FromGitter>
<tenebrousedge> so the other new code is not executing?
<FromGitter>
<ArtLinkov> Yeah, when I edit an existing module it just uses the code without the changes
<FromGitter>
<tenebrousedge> you put a `foo` method in there, and its `foo` test fails?
<FromGitter>
<ArtLinkov> OMG I knew it was something stupid! ⏎ Turns out I had another file with the same old code in another folder and the code was running from it (for some reason, I don't remember putting it there T.T)
<FromGitter>
<ArtLinkov> Thanks for being my rubber duck @tenebrousedge :)
<FromGitter>
<tenebrousedge> np
<FromGitter>
<dubesoftware> @tenebrousedge What font is that?
<FromGitter>
<tenebrousedge> font?
<FromGitter>
<dubesoftware> Yep, in your code sample.
<FromGitter>
<bajro17> :) its gitter font for code
<FromGitter>
<tenebrousedge> ^
<FromGitter>
<tenebrousedge> DejaVu Sans Mono says the inspector
<FromGitter>
<meltheadorable> my inspector just says `monospace`
<FromGitter>
<bajro17> my dont show any font :D
<FromGitter>
<bajro17> I find correct one bitstream vera sans mono
<FromGitter>
<tenebrousedge> probably system default
<FromGitter>
<dubesoftware> You got it @Melody, ta. Works in VS Code. ⏎ ⏎ It’s a Mono family font for sure. Now to set the weight...
<FromGitter>
<bajro17> I wish I can same like @watzon find ideal job :'( and work in crystal I have long experience with golang but crystal have something magical inside I every time come back
<FromGitter>
<dubesoftware> It’s Courier
<FromGitter>
<tenebrousedge> it's what your system default is
<FromGitter>
<dubesoftware> Back to Menlo. “If it ain’t broke…"
<FromGitter>
<absolutejam_gitlab> So
<FromGitter>
<absolutejam_gitlab> Say I wanted to dynamically generate some classes
<FromGitter>
<absolutejam_gitlab> from a yaml file
<FromGitter>
<absolutejam_gitlab> and then use this as a lib
<FromGitter>
<absolutejam_gitlab> thinking out loud...
<FromGitter>
<absolutejam_gitlab> just wondering how I can read yaml at compile time
<FromGitter>
<absolutejam_gitlab> or do I just read it normally...
<FromGitter>
<absolutejam_gitlab> I always struggle with metaprogramming compile time vs run time
<FromGitter>
<Blacksmoke16> gl
<FromGitter>
<Blacksmoke16> i dont think its possible
<FromGitter>
<tenebrousedge> you have no idea what the classes will be?
<FromGitter>
<absolutejam_gitlab> well I'm trying to create classes based on an openapi spec
<FromGitter>
<Blacksmoke16> it would be possible if you define classes, then use `from_yaml`
<FromGitter>
<Blacksmoke16> sec i can give you an example
<FromGitter>
<meltheadorable> i think you’d want to do code generation rather than metaprogramming for this — dump to actual files and then compile
<FromGitter>
<absolutejam_gitlab> `Modules and classes`
<FromGitter>
<absolutejam_gitlab> there was no anchor to link to
<FromGitter>
<tenebrousedge> all classes and methods must exist at compile time; it's not possible to create them at runtime
<FromGitter>
<absolutejam_gitlab> Next question - Can I read yaml at compile time? haha
<FromGitter>
<tenebrousedge> in theory <_<
<FromGitter>
<meltheadorable> What is your end goal? Like what are you trying to read the OpenAPI spec to produce?
<FromGitter>
<absolutejam_gitlab> I want to consume the Docker openapi spec
<FromGitter>
<absolutejam_gitlab> to wrap the API
<FromGitter>
<meltheadorable> is there a reason you can’t just do code generation for this, dump the files, like people are doing with auto-generated bindings?
<FromGitter>
<absolutejam_gitlab> I've never even attempted anything like this, so this was my first idea
<FromGitter>
<meltheadorable> i mean that’s basically what you’re doing, is generating bindings for a JSON API
<FromGitter>
<absolutejam_gitlab> Do you mean generate the actual files on disk instead of the ASTNodes in the files
<FromGitter>
<absolutejam_gitlab> It may be easier, haha
<FromGitter>
<meltheadorable> you can create an OpenAPI code generator that reads the specs and generates actual crystal files in a folder, you then treat those as a separate project to consume the API with
<FromGitter>
<meltheadorable> the same way autogenned bindings work
<FromGitter>
<meltheadorable> and it lets you change/enhance them if you need to
<FromGitter>
<absolutejam_gitlab> I hadn't even thought about that
<FromGitter>
<meltheadorable> so you’ve got the application that reads OpenAPI specs and generates crystal code
<FromGitter>
<absolutejam_gitlab> Super meta
<FromGitter>
<absolutejam_gitlab> I like it
<FromGitter>
<Blacksmoke16> you can read files at compile time but it just gives you a string literal
<FromGitter>
<meltheadorable> and then anyone who wants to generate an API consumer for an OpenAPI based library can use your project to create their first shard
<FromGitter>
<meltheadorable> and iterate on the output
<FromGitter>
<tenebrousedge> #myfirstshard :D
<FromGitter>
<meltheadorable> like if there’s things that are more complicafted or can’t be represented in the spec fully
<FromGitter>
<meltheadorable> or they just want a nicer user-facing API for it
<FromGitter>
<meltheadorable> but yeah you basically generate the kind of skeletal building blocks for the API consumer as a project
<FromGitter>
<meltheadorable> rather than trying to dynamically create all of it in memory and then use that code every time
<FromGitter>
<absolutejam_gitlab> yeah, makes sense honestly
<FromGitter>
<absolutejam_gitlab> a 2 step process
<FromGitter>
<meltheadorable> there’s value in being able to consume a spec on the fly and then interface with the generated version at runtime, but if you’re just looking to produce a consumer once without manually writing it all, generating the code as files makes sense
<FromGitter>
<meltheadorable> that said i think @Blacksmoke16 is right that there’s another approach here where you just define the types of things that OpenAPI knows about, so you’d create like `OpenAPI::Entity` or `OpenAPI::Property` classes and then you could do something like ⏎ ⏎ ```client = OpenAPI::Client.from_yaml(“spec.yml”) ⏎ response = client[“PetShop”].get(“index”)``` ⏎ ⏎ which would look for
<FromGitter>
... the `OpenAPI::Entity` with the name `PetShop`and then `get` its `index` endpoint, which could all be done without any meta programming, because they’d just be instances of these assorted known object types [https://gitter.im/crystal-lang/crystal?at=5d405e6f2316544a6df33fa3]
<FromGitter>
<Blacksmoke16> mhm
<FromGitter>
<Blacksmoke16> benefit of having a spec
<FromGitter>
<meltheadorable> an oversimplification but i think you get the idea
<FromGitter>
<Blacksmoke16> would prob want to add methods to it to make the API better
<FromGitter>
<Blacksmoke16> vs `client.paths.get.200...`
<FromGitter>
<Blacksmoke16> structs would be perfect for this as well since the data is immutable
<FromGitter>
<meltheadorable> i think for your use case, the code generation approach is probably more what you’re after, not sure what the use case is for generic OpenAPI clients, i guess people with really simple use cases could prototype with them instead of wasting their time with writing a whole shard, or use it under the hood to make it easier to update later
<FromGitter>
<meltheadorable> but the interface is gonna be necessarily less natural
<FromGitter>
<Blacksmoke16> id be interested to see how it goes, was looking to implement something similar for Athena a while ago. where you could generate the spec based on your defined routes/types and such
<FromGitter>
<meltheadorable> dunno
laaron has quit [Remote host closed the connection]
dostoyevsky has quit [Quit: leaving]
dostoyevsky has joined #crystal-lang
<allan>
@absolutejam_gitlab is crystal your first compiled language?
<allan>
normally with compiled languages, you don’t attempt to do things “at compile time” (other than templating mechanisms, which are more akin to macros)
<allan>
but I can see coming from a scripting language that the equivalent to meta programming would be compile-time stuff
<allan>
because for a scripting language, generating source files versus code in memory seems to be about the same, with the latter probably being easier
<allan>
but for compiled languages, it’s the opposite
<allan>
and once you’re anyway compiling your source files, you tend to have a build system that can take care of all auto-generated/preprocessed stuff
rohitpaulk has joined #crystal-lang
rohitpaulk has quit [Remote host closed the connection]
duane has quit [Ping timeout: 248 seconds]
ua has quit [Ping timeout: 244 seconds]
ua has joined #crystal-lang
rohitpaulk has joined #crystal-lang
duane has joined #crystal-lang
<FromGitter>
<absolutejam_gitlab> Sorry, just back at my computer
<FromGitter>
<absolutejam_gitlab> It's my 2nd foray - the first being Nim - but I've never done an awful lot with them
<FromGitter>
<absolutejam_gitlab> and toyed briefly with golang
rohitpaulk has quit [Remote host closed the connection]
duane has quit [Ping timeout: 244 seconds]
<FromGitter>
<tenebrousedge> it's weird coming back to the trumpet after like, twenty years. I played in junior high, but briefly -- I wasn't that great, and there were too many trumpets, so I played baritone and trombone through high school. I made All-State on those, but in Alaska that wasn't much of an accomplishment. I bought a trumpet when I was living in Panama four years ago, but haven't had practice space until a few days
<FromGitter>
... ago. I don't have a lot of stamina, my tone is muddy, dynamic range is not great, but I remember not being able to play above the staff at all, and that doesn't seem to be a problem any more. It's very strange to be better now than I was then
<FromGitter>
<kinxer> @tenebrousedge That's really cool. Keep it up! I did baritone in middle and high school, and a little in college, but I don't play it much anymore.
<FromGitter>
<tenebrousedge> I keep reading articles that say that playing music is one of the best ways to keep one's brain young. It's probably too late for that one 😉 but it's fun to play, and we'll see what comes of it
<FromGitter>
<watzon> I feel like Crystal needs a Timeout module
<FromGitter>
<watzon> Ohhh I see, you're using serializers and want to access properties as if the object were a hash
<FromGitter>
<absolutejam_gitlab> parsing the openapi spec and that means a property `get`,`put`, etc. Maybe I should make a `hash` method that returns them as a hash
<FromGitter>
<absolutejam_gitlab> then I can iterate over the keys
<FromGitter>
<watzon> You can convert the object to a hash and iterate over the keys that way
<FromGitter>
<tenebrousedge> make it `to_h`, not `hash`
<FromGitter>
<absolutejam_gitlab> yay
<FromGitter>
<absolutejam_gitlab> and yeah `to_h` is idiomatic, thanks
<FromGitter>
<tenebrousedge> well, and `Object#hash` is already a thing
<FromGitter>
<absolutejam_gitlab> i've been taking inspiration from that
<FromGitter>
<watzon> Yeah `hash` is something totally different
<FromGitter>
<absolutejam_gitlab> and then found that they do that
teardown has joined #crystal-lang
<FromGitter>
<meltheadorable> fwiw yall i managed to get a test running that correctly starts my SMTP server and then connects to it with a TCPSocket and send commands, I don’t really understand how it works and I’m gonna need to rewrite all this about 10 times but its a start
<FromGitter>
<Blacksmoke16> \o/
<FromGitter>
<watzon> woo hoo!
<FromGitter>
<meltheadorable> yeah but now i kinda need to step back and figure out like
<FromGitter>
<meltheadorable> overall API design
<FromGitter>
<meltheadorable> unless i make this my whole mail server instead of an SMTP library and try to extract the library later I’m gonna need like, some way to hook into the server provide alternative authenticators (if you are supporting auth, you need something to compare the credentials to) or handlers for what to do with actual messages — and @Blacksmoke16 do not said “dependency injection” getting alternative
<FromGitter>
... authenticators into the spec is the easy part, the hard part is finding a way to make the design flexible enough that extensions can define valid commands and *where* they are valid, and control their sequence/pass control back to the main SMTP resolver — because some commands are only valid in some contexts because SMTP is a stateful ... [https://gitter.im/crystal-lang/crystal?at=5d40d2572316544a6df34057]
<FromGitter>
<meltheadorable> which means extensions will need to be able to merge information about its states, if not its substates, with the main server implementation
<FromGitter>
<meltheadorable> kind of an architecture heading & im feeling the need to re-read like several books on design patterns
<FromGitter>
<Blacksmoke16> sounds like you got your work cut out for you
<FromGitter>
<meltheadorable> yeah for sure
<FromGitter>
<Blacksmoke16> one pattern i use a lot (not really sure if it would fit this case tho)
<FromGitter>
<Blacksmoke16> if you have a parent class and have each alternate authenticator inherit from it
<FromGitter>
<Blacksmoke16> you can get an array of them at compile time via macros
<FromGitter>
<Blacksmoke16> then use annotations on the subclass to store data
<FromGitter>
<meltheadorable> may re-shelve this project for a while
<FromGitter>
<Blacksmoke16> design is usually the hardest part
<FromGitter>
<meltheadorable> I only need to support one authenticator at a time, the problem is dealing with the command registration and response flow, and dealing with extensions that steal control from the main loop entirely, to accept like, a bunch of buffered input.
<FromGitter>
<meltheadorable> Hard to let those behaviors be user-defined
<FromGitter>
<Blacksmoke16> do they have to be?
<FromGitter>
<meltheadorable> I can special-case a few of them that don’t require interfacing with any kind of external stores, and just make those extensions library features
<FromGitter>
<meltheadorable> but some stuff has to provide new commands i won’t know about and also interface with other data sources i don’t know about, so library consumers will need to be able to define that sort of thing
<FromGitter>
<watzon> Imagine what I'm going through right now. I'm trying to recreate something like Puppeteer, but in Crystal
<FromGitter>
<watzon> 😦
<FromGitter>
<meltheadorable> I don’t have a good enough knowledge yet of SMTP extensions and mechanisms to know how many of them it makes sense to just deal with natively and how many will require external data sources or that people will need to implement as some kind of plugin
<FromGitter>
<meltheadorable> I was hoping to tackle this problem a little less all at once so was hoping I didn’t need to start with encyclopedic knowledge of the whole mess
<FromGitter>
<meltheadorable> but i suppose version 2.0 could always be rearchitected to support plugins and i could just ignore it all for now
<FromGitter>
<meltheadorable> it’s not like anyone but me wants to write SMTP servers anyway so i’m not gonna have any users except myself